Compare commits

..

24 Commits

Author SHA1 Message Date
d66f2c716c docs: env 示例补充 DAILY_WECOM_TOP_LINE / DAILY_FEATURED_REASON
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 17:08:35 +08:00
d9aef4b340 chore: 删 tmp 调试文件,裸 except 改命名异常+日志
- 删除被跟踪的 tmp_check_desc.py / tmp_desc_snip.txt
- bridge_manager discovery、generate release 抓取的裸
  except Exception 改命名异常 + logger.warning(T8)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 17:01:20 +08:00
d7992e7a7a feat: push_gate 判定分支加结构化日志
force/推送/静默日/无更新仍推 四分支各记一条 logger.info,
无人值守时可定位「早报没推是静默日还是别的原因」(T7)。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 16:55:47 +08:00
3321a307a0 feat: 首推理由行加 DAILY_FEATURED_REASON 开关
pick_why 本已渲染,现加开关默认开,空则省略(T6)。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 16:53:19 +08:00
33bfed0e79 feat: theme_line 取数升级,无评分命中时回退 theme_names
新增 _top_line: 优先评分最高主题,无命中时回退 _theme_clusters 的
主题名(非 markdown 示例),DAILY_WECOM_TOP_LINE=0 退回原逻辑。
复用现有 theme_line 行,不新增头部行(T5)。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 16:44:15 +08:00
ea8de9fe61 test: 新增 generate_report 黄金/确定性回归
冻结 _now_cst、mock 网络/文件/LLM(featured_pick 与 push_gate
有独立测试,此处冻结),验证同一输入下完整版与企微版产物逐字节
一致——拆分 _collect/_select/_render 不得改变行为(T1+T4)。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 16:35:42 +08:00
cd92644be4 refactor: 拆 generate_report 为 _collect/_select/_render 三段
所有选择逻辑(9 处 board_select + featured_pick + _localize + pad)
全归 _select,_render 纯拼装写盘,薄壳 generate_report 仅编排。
结构化 bundle 传参;副作用(record_pushed_links/_save_snapshot/
save_json)保持原时序在 _render 末尾。行为不变,89 测试绿(T2)。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 14:02:32 +08:00
92e0ed9a27 refactor: _theme_clusters 迁入 narrative_axis 并以注入消除反向依赖
theme_clusters/theme_names 经参数注入 THEME_RULES 与 skill_id_fn,
避免 generate.py 的循环 import(T3,为拆分 generate_report 铺路)。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-18 13:49:37 +08:00
6ea2a4e4c6 feat: 早报系统重构与功能增强
- 新增常驻调度器 daily/scheduler.py + run-scheduler.ps1(定时生成/推送)
- 新增 daily/bridge_manager.py:Windows 兼容的 Cursor SDK 桥接
- 新增 skills/daily-featured-pick 首推 Skill 与叙事轴/去重逻辑
- 新闻抓取窗口、GitHub 搜索、企微 delta 模式等多项改进
- 补充设计文档与 superpowers 计划/规范
- 新增对应测试(scheduler、featured_pick、github_search、news_fetch_window 等)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 18:12:00 +08:00
6192dd4e2a chore: 移除独立企微 bot/浏览器桥接系统
两套系统并存,bot/(browser_service、preview_service、wecom_media、
router、xiaobao 场景等)与早报生成是独立的一条线,不再维护,整体删除。
同时将 __pycache__/*.pyc/.cursor 加入 .gitignore。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 18:11:11 +08:00
4a128b0fa6 fix: 关闭新闻放宽凑数并剥离放宽窗口文案
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-14 10:42:00 +08:00
dcd0608b5d feat: 代码选定叙事轴并注入 Agent 开场约束
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-14 10:40:14 +08:00
54f164cdbf feat: 首推与昨日冲突时改推并保证一月不重复
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-14 10:39:12 +08:00
d448002e7a feat: generate 以 board_select 为唯一列表主人并写回 shown keys
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-14 10:36:53 +08:00
0c324f9ace feat: 实现 board_select 周去重与深池补满
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-14 10:32:39 +08:00
f563239e0b feat: 拆分 wecom_shown_keys 与 movement_baseline 历史层
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-14 10:31:49 +08:00
02e97e057f docs: 更新 Agent 技能与 env 示例以支持 Delta 模式
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-09 11:37:55 +08:00
3ad2e7c090 feat: 静默日跳过企微 webhook 推送
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-09 11:37:55 +08:00
588def8eb2 feat: 在 generate 流程中接入 Delta 模式与推送闸门
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-09 11:37:54 +08:00
b2dd8721d0 feat: 新增企微早报推送闸门判定逻辑
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-09 11:35:49 +08:00
e3b5623860 feat: 新增企微 Delta 榜单区块渲染与替换逻辑
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-09 11:35:49 +08:00
2f1fac9308 feat: 实现 Skills 跨榜去重与首日 baseline 模式判定
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-09 11:33:36 +08:00
ba8631c867 feat: 新增企微已推送新闻 link 去重缓存
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-09 11:33:26 +08:00
aff75141cb feat: 新增企微 Delta 模式相关配置读取函数
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-09 11:32:38 +08:00
115 changed files with 9429 additions and 5002 deletions

View File

@@ -1,18 +0,0 @@
.git
.github
.env
.env.local
output
.cache
logs
bot/.env
bot/.venv
bot/.cache
__pycache__
*.pyc
.idea
tmp_*
tests
.pytest_cache
*.md
!skills/**/*.md

View File

@@ -1,4 +1,4 @@
# 企微群机器人 webhook早报推送,与 bot API 模式凭证不同 # 企微群机器人 webhook早报推送
WECOM_WEBHOOK_KEY=your-webhook-key WECOM_WEBHOOK_KEY=your-webhook-key
# 早报内容 # 早报内容
@@ -8,6 +8,10 @@ DAILY_GITHUB_TRENDING_LIMIT=10
DAILY_GITHUB_EMERGING_LIMIT=10 DAILY_GITHUB_EMERGING_LIMIT=10
DAILY_GITHUB_TOPIC_LIMIT=10 DAILY_GITHUB_TOPIC_LIMIT=10
# 企微版可选行(默认均开)
# DAILY_WECOM_TOP_LINE=1 # 顶部「今日主题」行,无评分命中时回退主题名
# DAILY_FEATURED_REASON=1 # 首推下「为什么值得点开」理由行
# GitHub Trending # GitHub Trending
GITHUB_TRENDING_SINCE=daily GITHUB_TRENDING_SINCE=daily
# GITHUB_TRENDING_LANGUAGE=python # GITHUB_TRENDING_LANGUAGE=python
@@ -25,14 +29,18 @@ GITHUB_TRENDING_SINCE=daily
# GITHUB_TOKEN=ghp_xxxxxxxxxxxxxxxx # GITHUB_TOKEN=ghp_xxxxxxxxxxxxxxxx
# GITHUB_API_ENRICH=1 # GITHUB_API_ENRICH=1
# 企微短版(各榜 Top N默认 10 # 企微短版(各榜 Top N默认 5周内不重复见 DAILY_BOARD_DEDUP_DAYS
DAILY_WECOM_TRENDING=10 DAILY_WECOM_TRENDING=5
DAILY_WECOM_HOT=10 DAILY_WECOM_HOT=5
DAILY_WECOM_GITHUB_TRENDING=10 DAILY_WECOM_GITHUB_TRENDING=5
DAILY_WECOM_GITHUB_EMERGING=10 DAILY_WECOM_GITHUB_EMERGING=5
DAILY_WECOM_GITHUB_TOPIC=10 DAILY_WECOM_GITHUB_TOPIC=5
DAILY_WECOM_AI_NEWS=10 DAILY_WECOM_AI_NEWS=10
DAILY_WECOM_CN_AI_NEWS=8 # research 模式额外技术类时讯条数(叠加在 AI 时讯精选之上)
DAILY_WECOM_AI_NEWS_TECH=5
DAILY_WECOM_CN_AI_NEWS=10
# 企微新闻摘要字数(句读/词边界截断,不加省略号)
# DAILY_WECOM_NEWS_DESC_LIMIT=72
# 企微 Skills 合并前扫描池大小(同 source 合并后仍凑满 Top N # 企微 Skills 合并前扫描池大小(同 source 合并后仍凑满 Top N
# DAILY_WECOM_SKILL_POOL=200 # DAILY_WECOM_SKILL_POOL=200
@@ -42,10 +50,37 @@ DAILY_WECOM_CN_AI_NEWS=8
DAILY_WECOM_CHUNK_BYTES=4096 DAILY_WECOM_CHUNK_BYTES=4096
# DAILY_WECOM_MAX_PARTS=5 # DAILY_WECOM_MAX_PARTS=5
# 企微列表模式delta=仅展示新入榜 | full=全量 Top 榜(回退)
DAILY_WECOM_MODE=delta
# Delta 模式下新入榜优先,不足时用当日 Top 榜补满各区块条数0=仅展示变化)
# 补榜时会排除近 N 天 baseline 已出现过的条目,避免周内重复(默认 7 天)
DAILY_WECOM_DELTA_PAD=1
# 补榜时从更大候选池选取默认展示条数×5至少 50
# DAILY_WECOM_PAD_POOL=50
# 无历史 data.json 时full=首日全量一次 | empty=列表为空
DAILY_DELTA_BASELINE_FALLBACK=full
# 推送闸门不满足时跳过 webhook仍写 output
DAILY_SKIP_PUSH_WHEN_SILENT=1
# DAILY_FORCE_PUSH=1
# 已推送新闻 link 去重天数
DAILY_NEWS_DEDUP_DAYS=7
# 常驻调度python -m daily schedule
DAILY_SCHEDULE_TZ=Asia/Shanghai
DAILY_SCHEDULE_GENERATE_AT=08:50
DAILY_SCHEDULE_PUSH_AT=09:00
# 编辑指定今日首推(可选):关键词,或 关键词|URL
# Python Step 0 检索 → featured.jsonAgent / classic 企微「今日首推」优先使用
# DAILY_FEATURED_PICK=gstack
# DAILY_FEATURED_PICK=gstack|https://github.com/you/gstack
# 国际 AI 时讯RSS见 daily/news/feeds.py # 国际 AI 时讯RSS见 daily/news/feeds.py
DAILY_AI_NEWS=1 DAILY_AI_NEWS=1
# 国内 AI 时讯RSS见 daily/news/feeds_cn.py # 国内 AI 时讯RSS见 daily/news/feeds_cn.pyresearch 模式下忽略
DAILY_CN_AI_NEWS=1 DAILY_CN_AI_NEWS=1
# AI 时讯来源rss=RSS 抓取 | research=Cursor SDK + deep-researchWebSearch
# DAILY_AI_NEWS_MODE=research
# 英文描述 → 简短中文DAILY_CURSOR_EDITOR=0 时生效) # 英文描述 → 简短中文DAILY_CURSOR_EDITOR=0 时生效)
# DAILY_ZH_DESC=1 # DAILY_ZH_DESC=1
# DAILY_ZH_DESC_BATCH=20 # DAILY_ZH_DESC_BATCH=20
@@ -59,14 +94,9 @@ DAILY_CN_AI_NEWS=1
# Agent 工作流(推荐:叙事化早报) # Agent 工作流(推荐:叙事化早报)
# DAILY_REPORT_MODE=agent # DAILY_REPORT_MODE=agent
# DAILY_LLM_PROVIDER=auto
# auto — agent 模式优先 Cursor否则 OpenAIclassic 优先 OpenAI
# openai — 强制 OpenAI 兼容 API
# cursor — 强制 Cursor SDK
# CURSOR_API_KEY=cursor_... # CURSOR_API_KEY=cursor_...
# CURSOR_MODEL=composer-2.5 # CURSOR_MODEL=composer-2.5
# CURSOR_TIMEOUT=600 # DAILY_CURSOR_CWD=d:\LY\diy\skills-hot-daily
# DAILY_CURSOR_CWD=.
# 新增榜对比(较昨日 Top15供 Agent 导语/signals列表展示 Top N # 新增榜对比(较昨日 Top15供 Agent 导语/signals列表展示 Top N
# DAILY_DELTA_COMPARE_DEPTH=15 # DAILY_DELTA_COMPARE_DEPTH=15
@@ -74,22 +104,20 @@ DAILY_CN_AI_NEWS=1
# DAILY_DELTA_LOOKBACK_DAYS=7 # DAILY_DELTA_LOOKBACK_DAYS=7
# DAILY_FULL_DESC_LIMIT=0 # DAILY_FULL_DESC_LIMIT=0
# DAILY_FULL_NEWS_SUMMARY_LIMIT=0 # DAILY_FULL_NEWS_SUMMARY_LIMIT=0
DAILY_AI_NEWS_HOURS=72 DAILY_AI_NEWS_HOURS=24
DAILY_AI_NEWS_PER_FEED=3 DAILY_AI_NEWS_PER_FEED=3
DAILY_AI_NEWS_PER_CATEGORY=5 DAILY_AI_NEWS_PER_CATEGORY=5
# 新闻排序 / 去重(Phase 2 # 多样性 / 去重(见 docs/superpowers/specs/2026-07-14-wecom-diversity-dedup-design.md
# 标题 Jaccard 相似度阈值0-95默认 55 表示 0.55 DAILY_BOARD_DEDUP_DAYS=7
# DAILY_NEWS_TITLE_SIM=55 # DAILY_BOARD_POOL_SIZE=50
# Agent 模式 LLM 输入池classic 仍用 DAILY_WECOM_* 条数) # DAILY_FEATURED_DEDUP_DAYS=30
# DAILY_AGENT_NEWS_POOL=40 # DAILY_THEME_BAN_DAYS=7
# DAILY_AGENT_CN_NEWS_POOL=30 # DAILY_NARRATIVE_AXIS_DAYS=3
DAILY_NEWS_BACKFILL=0
# RSS 源:编辑 config/feeds.yamlintl / cn缺失时回退内置默认 # 国际时讯:在 24h 滚动窗口基础上,不早于今日 0 点DAILY_AI_NEWS_TZ
# 单源失败重试次数(默认 3含首次请求 DAILY_AI_NEWS_FLOOR_TODAY=1
# DAILY_RSS_RETRY=3 # DAILY_AI_NEWS_TZ=Asia/Shanghai
# 内容过滤config/sensitive_words.yaml + DAILY_CONTENT_FILTER=1
# DAILY_CONTENT_FILTER=0
# Reddit RSS403/429 时在 Reddit 偏好设置 → RSS feeds 复制 user / feed 参数) # Reddit RSS403/429 时在 Reddit 偏好设置 → RSS feeds 复制 user / feed 参数)
# REDDIT_RSS_USER=your_username # REDDIT_RSS_USER=your_username

6
.gitignore vendored
View File

@@ -2,11 +2,7 @@
.env.local .env.local
logs/ logs/
.cache/ .cache/
bot/.env
bot/.venv/
bot/.cache/
output output
__pycache__/ __pycache__/
*.pyc *.pyc
.idea/ .cursor/
tmp_*

3
.idea/.gitignore generated vendored Normal file
View File

@@ -0,0 +1,3 @@
# Default ignored files
/shelf/
/workspace.xml

9
.idea/daily-robots.iml generated Normal file
View File

@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<module type="JAVA_MODULE" version="4">
<component name="NewModuleRootManager" inherit-compiler-output="true">
<exclude-output />
<content url="file://$MODULE_DIR$" />
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
</component>
</module>

2
.idea/misc.xml generated
View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?> <?xml version="1.0" encoding="UTF-8"?>
<project version="4"> <project version="4">
<component name="ProjectRootManager" version="2" default="true"> <component name="ProjectRootManager" version="2" languageLevel="JDK_21" default="true" project-jdk-name="21" project-jdk-type="JavaSDK">
<output url="file://$PROJECT_DIR$/out" /> <output url="file://$PROJECT_DIR$/out" />
</component> </component>
</project> </project>

8
.idea/modules.xml generated Normal file
View File

@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ProjectModuleManager">
<modules>
<module fileurl="file://$PROJECT_DIR$/.idea/daily-robots.iml" filepath="$PROJECT_DIR$/.idea/daily-robots.iml" />
</modules>
</component>
</project>

2
.idea/vcs.xml generated
View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?> <?xml version="1.0" encoding="UTF-8"?>
<project version="4"> <project version="4">
<component name="VcsDirectoryMappings"> <component name="VcsDirectoryMappings">
<mapping directory="$PROJECT_DIR$" vcs="Git" /> <mapping directory="" vcs="Git" />
</component> </component>
</project> </project>

View File

@@ -1,27 +0,0 @@
FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
TZ=Asia/Shanghai
WORKDIR /app
RUN apt-get update \
&& apt-get install -y --no-install-recommends tzdata \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY daily/ daily/
COPY shared/ shared/
COPY skills/ skills/
COPY config/ config/
COPY run-daily.sh .
RUN chmod +x run-daily.sh \
&& mkdir -p output .cache logs
VOLUME ["/app/output", "/app/.cache", "/app/logs"]
ENTRYPOINT ["./run-daily.sh"]

241
README.md
View File

@@ -1,149 +1,164 @@
# daily-robots # skills-hot-daily
> **Daily Briefing** · **WeCom Push** · **Agent Mode** Skills / GitHub 早报推送(企微 Webhook
给 AI 工程小团队推送**每日 curated 情报**skills.sh 榜单、GitHub AI 趋势、国际/国内 RSS生成叙事化早报并推到企业微信群。 ## 项目结构
与 TLDR AI / Ben's Bites 不同,这里专注 **Agent Skills 生态** + **可 fork 的自托管流水线**(不是又一个 email newsletter ```
skills-hot-daily/
**产出示例**[docs/sample-output/wecom-agent-sample.md](docs/sample-output/wecom-agent-sample.md)Agent 模式企微版节选) ├── README.md
├── .env.example # 早报 webhook、GitHub 等
├── requirements.txt # Python 依赖
├── run-daily.ps1 # 生成 + 推送一条龙
├── send-wecom.ps1 # 仅推送
├── daily/ # 早报 Python 包
│ ├── __main__.py # python -m daily [generate|push]
│ ├── config.py
│ ├── generate.py
│ ├── report_data.py # JSON 中间层
│ ├── cursor_editor.py # Cursor 编辑层
│ ├── llm_client.py
│ ├── localize.py # 仅中文化Tier A
│ ├── format_wecom.py
│ ├── webhook.py
│ ├── news/ # 国际 AI 时讯 RSS
│ │ ├── feeds.py
│ │ └── fetch.py
│ └── github/
│ ├── auth.py
│ ├── search.py
│ └── trending.py
├── output/ # YYYY-MM-DD.md / .wecom.md / .data.json / .editorial.json
├── skills/daily-editor/ # 早报 Cursor 编辑 Skill
│ └── SKILL.md
├── logs/
└── .cache/
```
--- ---
## Quick Start ## 早报推送
1. **Clone** 本仓库 ```powershell
2. **安装依赖**`pip install -r requirements.txt` cd d:\LY\diy\daily-robots
3. **配置环境**`copy .env.example .env` pip install -r requirements.txt
4. **填写 webhook**:企微群 → 群机器人 → 添加 → 把 `key=` 写入 `.env` copy .env.example .env
```env .\run-daily.ps1
WECOM_WEBHOOK_KEY=your-webhook-key ```
```
5. **生成并推送**`.\run-daily.ps1`
生成文件在 `output/` - 生成:`output/YYYY-MM-DD.md`(完整版)、`output/YYYY-MM-DD.wecom.md`(企微短版)
- 仅生成:`.\run-daily.ps1 -SkipPush`
- 仅推送:`python -m daily push output\2026-06-25.wecom.md`
**Webhook 配置**:企微群 → 群机器人 → 添加,将 `key=` 后的值写入项目根 `.env`
```env
WECOM_WEBHOOK_KEY=your-key
```
**GitHub 数据源**(见 `.env.example`
| 来源 | 说明 |
|------|------|
| GitHub Trending | `GITHUB_TRENDING_MODE=scrape``api` |
| 新兴 / Topic | Search API`GITHUB_TOKEN` |
| Release | 可选 `GITHUB_REPOS=owner/repo` |
**国际 AI 时讯**RSS`daily/news/feeds.py`
| 类别 | 覆盖 |
|------|------|
| 厂商官方 | Anthropic、OpenAI、Google、Meta、Microsoft、Mistral、Cursor 等 |
| Agent / LLM 开发者 | LangChain、LlamaIndex、Hugging Face、Copilot 等 |
| 综合科技媒体 | The Verge、TechCrunch、Ars、Wired、MIT TR 等 |
| Newsletter | Ben's Bites、Latent Space、Simon Willison、TLDR AI 等 |
| 研究 / 论文 | arXiv cs.CL/AI/LG、HF Papers |
| 社区讨论 | HN、Reddit r/LocalLLaMA / ClaudeAI / ML 等 |
环境变量:`DAILY_AI_NEWS=1` · `DAILY_CN_AI_NEWS=1` · `DAILY_AI_NEWS_HOURS=24` · `DAILY_WECOM_AI_NEWS=10` · `DAILY_WECOM_CN_AI_NEWS=10`
**国内 AI 时讯**RSS`daily/news/feeds_cn.py`
| 类别 | 覆盖 |
|------|------|
| AI 专业媒体 | 量子位 |
| 综合科技 | 36氪、雷锋网、Google News 中文 |
### 生成架构Tier B · Cursor 编辑层)
早报默认走 **Python 抓取 + 模板渲染**;可选开启 Cursor 做「编辑」:
```
抓取数据 → output/日期.data.json → Cursor 读 Skill 写 editorial → 模板填字 → .md / .wecom.md
```
| 文件 | 说明 | | 文件 | 说明 |
|------|------| |------|------|
| `YYYY-MM-DD.wecom.md` | 企微推送版(主产物 | | `output/YYYY-MM-DD.data.json` | 结构化榜单(供 LLM 输入 |
| `YYYY-MM-DD.md` | 完整归档版 | | `output/YYYY-MM-DD.editorial.json` | Cursor 输出的主题、速览、中文描述 |
| `YYYY-MM-DD.data.json` | 结构化数据(供 LLM / 调试 | | `skills/daily-editor/SKILL.md` | 编辑规范语气、JSON 格式 |
常用变体: ```env
# 开启 Tier B需 CURSOR_API_KEY 或 DAILY_LLM_API_KEY
```powershell DAILY_CURSOR_EDITOR=1
.\run-daily.ps1 -SkipPush # 只生成
python -m daily push output\2026-07-03.wecom.md # 只推送
``` ```
Linux / macOS 等价脚本: - 开启后:**一次 LLM 调用** 生成 `theme_line` + `highlights` + 全部中文描述
- 关闭时(默认):规则主题 + `DAILY_ZH_DESC` 仅中文化描述
- LLM 失败自动回退规则模式,不影响推送
```bash ### 生成架构Agent 工作流 · 推荐)
chmod +x run-daily.sh
./run-daily.sh # 生成 + 推送 若觉得模板版「榜单堆砌」不友好,可改用 **Agent 三步流水线**
./run-daily.sh --skip-push # 只生成
./run-daily.sh --force # 忽略 30 分钟内重复运行锁 ```
Python 抓取 → Step1 趋势分析 → Step2 叙事写稿 → Python 分条推送企微
(.trends.json) (.wecom.md)
``` ```
**前置条件**Python 3.10+、企业微信群机器人 webhook。无需 Bot API 凭证。 | 模式 | 环境变量 | 企微版风格 |
|------|----------|------------|
--- | `classic`(默认) | — | 分区榜单 + 模板 |
| `editor` | `DAILY_CURSOR_EDITOR=1` | 模板 + LLM 中文化 |
## 部署Docker | `agent` | `DAILY_REPORT_MODE=agent` | **导语 + 信号 + 精选**,全中文叙述 |
```bash
cp .env.example .env # 填入 WECOM_WEBHOOK_KEY 等
docker compose build
docker compose run --rm daily # 生成 + 推送
docker compose run --rm daily --skip-push # 只生成
```
`output/`、`.cache/`、`logs/` 挂载到宿主机,与本地 `run-daily.ps1` 行为一致。
---
## Agent 模式(推荐)
默认 `classic` 是分区榜单模板;**推荐 Agent 模式**——导语 + 今日信号 + 精选新闻 + 榜单,全中文叙述。
```env ```env
DAILY_REPORT_MODE=agent DAILY_REPORT_MODE=agent
CURSOR_API_KEY=cursor_... CURSOR_API_KEY=cursor_...
CURSOR_MODEL=composer-2.5 CURSOR_MODEL=composer-2.5
DAILY_CURSOR_CWD=. DAILY_CURSOR_CWD=d:\LY\diy\daily-robots
``` ```
| 模式 | 环境变量 | 风格 | | 文件 | 说明 |
|------|----------|------| |------|------|
| `classic` | — | 分区榜单 + 模板 | | `output/YYYY-MM-DD.trends.json` | Step1 趋势分析结果 |
| `editor` | `DAILY_CURSOR_EDITOR=1` | 模板 + LLM 中文化 | | `skills/daily-agent/SKILL.md` | Agent 工作流规范 |
| **`agent`** | `DAILY_REPORT_MODE=agent` | **叙事化早报(推荐)** |
流水线: - 完整版 `YYYY-MM-DD.md` 仍为数据表格归档;企微版由 Agent 直接写 Markdown
- Agent 失败自动回退 `classic`,不影响 `run-daily.ps1`
``` 定时推送:
Python 抓取 → Step1 趋势分析 (.trends.json) → Step2 写企微稿 (.wecom.md) → webhook 分条推送
```
- 规范见 `skills/daily-agent/SKILL.md` - **常驻调度(推荐)**`python -m daily schedule``.\run-scheduler.ps1`(默认 08:50 生成、09:00 推送,见 `DAILY_SCHEDULE_*`
- LLM 失败自动回退 `classic`,不影响 `run-daily.ps1` - Windows 任务计划:`.\register-daily-task.ps1`
- 无 `CURSOR_API_KEY` 时请用 `classic`,或配置 `DAILY_LLM_API_KEY`OpenAI 兼容 API - Cursor`/loop 1d`(时间会漂移,仅临时用
定时推送Windows 任务计划程序每日执行 `run-daily.ps1`,或 cron / `schtasks`。
--- ---
## 自定义 RSS 源 ## 其他推送方案
RSS 源配置在 **`config/feeds.yaml`**(国际 `intl` / 国内 `cn`),改 URL 或增删 feed 后无需改 Python。文件缺失或解析失败时会回退 `daily/news/feeds_defaults.py` 内置列表。 | 方案 | 适用场景 | 复杂度 |
|------|----------|--------|
可选 **`config/sensitive_words.yaml`** + `DAILY_CONTENT_FILTER=1`:标题/摘要命中敏感词则丢弃该条。 | **群机器人 webhook** | 推送到固定群 | 低 |
| **应用消息 API** | 推送给指定成员/部门 | 中(需 corp_id、secret、agent_id |
旧版硬编码路径(仍可读作参考):`daily/news/feeds_defaults.py` | **邮件 + 企业微信邮箱** | 已有 SMTP | 中 |
| **PushPlus / Server酱** | 个人微信中转 | 低(第三方) |
```env
DAILY_AI_NEWS=1
DAILY_CN_AI_NEWS=1
DAILY_AI_NEWS_HOURS=72
DAILY_WECOM_AI_NEWS=10
DAILY_WECOM_CN_AI_NEWS=8
```
Skills 榜单默认爬 skills.sh 官网600+ 条);可改为第三方 feed
```env
# SKILLS_BOARD_SOURCE=feed
```
GitHub 新兴/Topic 榜需 `GITHUB_TOKEN`(见 `.env.example` 注释)。
---
## Experimental`bot/` 企微对话机器人
同仓库内的 **API 模式智能机器人**@ 机器人快查 skills、Cursor 任务、Playwright 截图)**不在 v1 开源支持范围内**,需独立 venv 与 `WECOM_BOT_ID` / `SECRET`。
详见 **[bot/README.md](bot/README.md)**。
---
## 项目结构
```
daily-robots/
├── run-daily.ps1 # 生成 + 推送
├── daily/ # 早报主包
├── shared/skills_data.py # skills feed 共用数据层
├── skills/daily-agent/ # Agent 工作流规范
├── output/ # 生成产物gitignore
└── bot/ # experimental
```
--- ---
## 注意事项 ## 注意事项
- Webhook / API Key **勿提交 Git**,只用 `.env` - Webhook **不要提交 Git**,只用环境变量
- 企微 markdown 为子集;超长报告**自动分条推送**(默认 4096 bytes/条 - 企业微信 markdown 为**子集**(不支持完整 GitHub 表格语法时可改为文本列表
- 完整配置项见 [`.env.example`](.env.example) - 单条消息约 **4096 字节** 上限,`send-wecom.ps1` 已做截断
- 开发测试:`pip install -r requirements-dev.txt` → `pytest`

View File

@@ -1,29 +0,0 @@
# 企业微信智能机器人API 模式 · 长连接)
# 管理后台 → 安全与管理 → 管理工具 → 智能机器人 → 创建 → API 模式 → 使用长连接
WECOM_BOT_ID=your-bot-id
WECOM_BOT_SECRET=your-bot-secret
# Cursor SDK@ 机器人后的通用任务由 Cursor 执行)
CURSOR_API_KEY=cursor_...
CURSOR_CWD=.
CURSOR_MODEL=composer-2.5
CURSOR_TIMEOUT=600
# 前端截图预览(基于 CURSOR_CWD
PREVIEW_PORT=5173
PREVIEW_URL=http://127.0.0.1:5173/
# PREVIEW_DEV_COMMAND=npm run dev
# PREVIEW_STARTUP_TIMEOUT=120
# 登录后截图(账号密码只放 .env切勿发到企微群
# PREVIEW_LOGIN_USER=your_account_or_phone
# PREVIEW_LOGIN_PASSWORD=your_password
# PREVIEW_AFTER_LOGIN_URL=/app/dashboard
# PREVIEW_AUTO_LOGIN=true
# 网页操作场景目录(可选,默认 bot/scenarios 与 CURSOR_CWD/.browser-scenarios
# BROWSER_SCENARIOS_DIR=d:\path\to\scenarios
# BROWSER_DEFAULT_SCENARIO=xiaobao-agent-manage
# hybrid=快查走本地 / 其余走 Cursor | cursor=全部 Cursor | skills=仅本地
ROUTING_MODE=hybrid

4
bot/.gitignore vendored
View File

@@ -1,4 +0,0 @@
.cache/
.env
.venv/
.cache/screenshots/

View File

@@ -1,143 +0,0 @@
# bot/ — 企微 API 模式智能机器人
> **EXPERIMENTAL** — 本模块不在 daily-robots v1 开源支持范围内。
> 主产品为根目录 **早报推送**webhook见 [README](../README.md)。
> Bot 需独立 venv、企微 API 凭证、常驻进程;问题请自行排查或提 issue 标注 `bot`。
在企微里 @ 机器人即可:
- **快查**`trending 10``hot 10``搜索 react`(本地 skills 数据,秒回)
- **截图预览**`preview` / `截图`(基于 `CURSOR_CWD` 启动前端并发图)
- **通用任务**:任意自然语言需求,由 **Cursor Agent** 执行并回传结果
数据层与早报共用 [`shared/skills_data.py`](../shared/skills_data.py)。
---
## 1. 创建 API 模式机器人
1. [企业微信管理后台](https://work.weixin.qq.com/) → **安全与管理****管理工具****智能机器人****创建机器人**
2. 选择 **API 模式创建****使用长连接**
3. 记录 **Bot ID****Secret**Secret 只显示一次,请立即保存)
4. 设置可见范围,将机器人 **添加到目标群** 或允许成员单聊
普通成员路径:工作台 → 智能机器人 → 手动创建 → API 模式 → 长连接
---
## 2. 启动本地服务
```powershell
cd path\to\daily-robots\bot
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
playwright install chromium
copy .env.example .env
# 编辑 .envWECOM_BOT_ID / WECOM_BOT_SECRET / CURSOR_API_KEY
python main.py
```
服务需 **常驻运行**(本机、服务器或 Docker。长连接模式下机器人进程须在线才能收消息。
---
## 3. 路由模式ROUTING_MODE
| 模式 | 行为 |
|------|------|
| `hybrid`(默认) | `trending`/`hot`/`搜索`/`详情` 走本地快查;其余 @ 消息交给 Cursor |
| `cursor` | 所有消息都交给 Cursor 执行 |
| `skills` | 仅本地 skills 快查 |
**Cursor 任务示例**(群里发送):
```
@test 总结 trending top10并推荐 3 个适合前端团队的 skill
@test 对比 mattpocock/skills 和 obra/superpowers 各有哪些热门 skill
@test 帮我写一段 npx skills add 的安装说明
```
Cursor 在本机 `CURSOR_CWD` 目录下运行(见 `bot/.env`)。复杂任务可能需要 110 分钟流式消息会显示「Cursor 正在执行任务…」。
---
## 4. 前端截图预览API 模式发图)
项目路径读取 `.env` 中的 **`CURSOR_CWD`**。机器人会:
1.`CURSOR_CWD` 检测 `package.json`,若有 `dev` 脚本则执行 `PREVIEW_DEV_COMMAND`(默认 `npm run dev`
2. 等待 `PREVIEW_PORT`(默认 `5173`)就绪,或用 `PREVIEW_URL` 直接访问
3. Playwright 打开页面并截图
4. 通过 API 模式 **上传图片 + 回复 image 消息** 到群
| 命令 | 说明 |
|------|------|
| `preview` / `截图` / `预览` | 访问 `http://127.0.0.1:5173/` 并截图 |
| `preview /login` | 指定路径 |
| `preview / 3000` | 指定端口 |
| `preview http://127.0.0.1:8080/` | 指定完整 URL |
`.env` 可选配置:
```env
CURSOR_CWD=.
PREVIEW_PORT=5173
PREVIEW_URL=http://127.0.0.1:5173/
PREVIEW_DEV_COMMAND=npm run dev
PREVIEW_STARTUP_TIMEOUT=120
```
---
## 4b. 网页操作Playwright 步骤引擎)
支持三种方式定义操作流程,**无需改 Python 代码**
**1. 自然语言(企微里直接说)**
```
@test 访问登录页,输入账号密码,点击登录后进入主页,点击智能体管理菜单然后截图
```
账号密码从 `.env` 读取(`{{PREVIEW_LOGIN_USER}}` / `{{PREVIEW_LOGIN_PASSWORD}}`),勿在群里发密码。
**2. 场景文件 YAML** — 见 `scenarios/`,触发:`@test browser <场景名>`
**3. 消息内 DSL** — 以 `browser:` 开头的多行步骤
支持的步骤:`goto` · `fill` · `click` · `wait` · `screenshot` · `press`
---
## 5. 快查命令
| 命令 | 说明 |
|------|------|
| `trending 10` / `趋势 10` | 近期增长榜 Top N默认 10最大 30 |
| `hot 10` / `实时 10` | 实时热度榜 |
| `all 10` / `总榜 10` | 历史总安装榜 |
| `搜索 react` / `search tdd` | 关键词搜索 |
| `详情 find-skills` | 单个 skill 详情 + 安装命令 |
| `preview` / `截图` | 启动 CURSOR_CWD 前端并截图发群 |
| `帮助` | 命令列表 |
---
## 6. 本地测试(无需企微凭证)
```powershell
cd path\to\daily-robots\bot
python -c "from skills_service import handle_command; print(handle_command('trending 5'))"
```
---
## 配置
| 文件 | 说明 |
|------|------|
| `bot/.env` | `WECOM_BOT_ID``WECOM_BOT_SECRET``CURSOR_API_KEY` 等 |
| `.env.example` | 模板 |
与根目录 `.env`(早报 webhook**相互独立**,勿混用。

View File

@@ -1,12 +0,0 @@
"""Bot 内部数据结构。"""
from __future__ import annotations
from dataclasses import dataclass
@dataclass
class RouteResult:
source: str
text: str
image_path: str | None = None

View File

@@ -1,49 +0,0 @@
"""浏览器场景变量替换与 base URL 解析。"""
from __future__ import annotations
import re
from urllib.parse import urlparse
import env_config
_VAR_PATTERN = re.compile(r"\{\{([A-Z0-9_]+)\}\}")
def interpolate(value: str) -> str:
def repl(match: re.Match[str]) -> str:
key = match.group(1)
resolved = env_config.env(key)
if resolved is None:
raise RuntimeError(f"场景变量未配置:{key}")
return resolved
return _VAR_PATTERN.sub(repl, value)
def default_base_url() -> str:
explicit = (env_config.env("PREVIEW_BASE_URL") or "").strip()
if explicit:
return interpolate(explicit.rstrip("/"))
preview = (env_config.env("PREVIEW_URL") or "").strip()
if preview:
parsed = urlparse(preview)
scheme = parsed.scheme or "http"
host = parsed.hostname or "127.0.0.1"
port = parsed.port
if port and port not in (80, 443):
return f"{scheme}://{host}:{port}"
return f"{scheme}://{host}"
port = env_config.env("PREVIEW_PORT", "5173") or "5173"
return f"http://127.0.0.1:{port}"
def resolve_url(base_url: str, target: str) -> str:
target = interpolate(target.strip())
if target.startswith("http://") or target.startswith("https://"):
return target
if not target.startswith("/"):
target = "/" + target
return base_url.rstrip("/") + target

View File

@@ -1,269 +0,0 @@
"""通用 Playwright 步骤执行器(不写死业务页面)。"""
from __future__ import annotations
import logging
import re
import time
from datetime import datetime
from pathlib import Path
from typing import Any
from browser_env import interpolate, resolve_url
from browser_models import BrowserResult, BrowserScenario
logger = logging.getLogger(__name__)
SCREENSHOT_DIR = Path(__file__).resolve().parent / ".cache" / "screenshots"
FIELD_HINTS: dict[str, list[str]] = {
"账号": [
"#login-username",
"input#login-username",
"input[autocomplete='username']",
"username",
"account",
"phone",
"账号",
"手机号",
"企业账号",
],
"密码": [
"#login-password input",
"#login-password",
"input#login-password",
"input[type='password']",
"password",
"密码",
],
"用户名": ["#login-username", "input#login-username", "username", "account", "账号"],
}
def _step_label(step: dict[str, Any], index: int) -> str:
action = step.get("action", "?")
target = step.get("target") or step.get("field") or step.get("url") or ""
return f"{index + 1}. {action} {target}".strip()
def _resolve_fill_locator(page, field: str, step: dict[str, Any]):
if step.get("selector"):
return page.locator(interpolate(str(step["selector"])))
field_key = interpolate(str(field))
if step.get("label"):
return page.get_by_label(interpolate(str(step["label"])), exact=False)
if step.get("placeholder"):
return page.get_by_placeholder(interpolate(str(step["placeholder"])), exact=False)
hints = FIELD_HINTS.get(field_key, [field_key])
for hint in hints:
if hint.startswith("#") or hint.startswith(".") or hint.startswith("["):
locator = page.locator(hint)
if locator.count() > 0:
return locator.first
for getter in (
lambda h=hint: page.get_by_label(h, exact=False),
lambda h=hint: page.get_by_placeholder(h, exact=False),
):
locator = getter()
if locator.count() > 0:
return locator.first
return page.locator("input, textarea").filter(has_text=field_key).first
def _fill_field(page, field: str, step: dict[str, Any]) -> None:
value = interpolate(str(step.get("value", "")))
locator = _resolve_fill_locator(page, field, step)
locator.click(timeout=10_000)
locator.fill("", timeout=5_000)
locator.fill(value, timeout=10_000)
def _page_error_text(page) -> str | None:
for selector in (
".ant-message-error",
".ant-form-item-explain-error",
".ant-alert-error",
):
try:
locator = page.locator(selector).first
if locator.is_visible(timeout=300):
text = locator.inner_text(timeout=1_000).strip()
if text:
return text
except Exception:
continue
return None
def _pathname_matches(pattern: str, pathname: str) -> bool:
pattern = pattern.strip()
if pattern in {"**/app/**", "**/app/*", "/app/**"}:
return pathname.startswith("/app")
if pattern.endswith("/**"):
prefix = pattern[:-3].rstrip("/")
if prefix.startswith("**/"):
prefix = prefix[3:]
if not prefix.startswith("/"):
prefix = "/" + prefix
return pathname.startswith(prefix)
if "**" in pattern or "*" in pattern:
regex = "^" + re.escape(pattern).replace(r"\*\*", ".*").replace(r"\*", "[^/]*") + "$"
return re.search(regex, pathname) is not None
return pathname == pattern or pathname.startswith(pattern)
def _wait_for_url_pattern(page, pattern: str, timeout: int = 60_000) -> None:
"""SPA 路由用 pathname 轮询glob 模式不依赖 navigation 事件。"""
deadline = time.monotonic() + timeout / 1000
last_error: str | None = None
while time.monotonic() < deadline:
pathname = page.evaluate("() => window.location.pathname")
if _pathname_matches(pattern, pathname):
try:
page.wait_for_load_state("networkidle", timeout=8_000)
except Exception:
page.wait_for_timeout(800)
return
err = _page_error_text(page)
if err and err != last_error:
last_error = err
logger.warning("页面提示:%s", err)
if "/login" in pathname:
raise RuntimeError(f"登录失败:{err}")
page.wait_for_timeout(400)
err = _page_error_text(page)
hint_parts = [f"当前 URL`{page.url}`"]
if err:
hint_parts.append(f"页面错误:{err}")
elif last_error:
hint_parts.append(f"页面错误:{last_error}")
hint_parts.append("请确认 PREVIEW_LOGIN_USER/PASSWORD 正确,且登录 API内网网关可达。")
raise RuntimeError(f"等待 URL 匹配 `{pattern}` 超时({timeout}ms{' '.join(hint_parts)}")
def _click_target(page, target: str) -> None:
target = interpolate(target.strip())
if target.lower() in {"登录", "login"}:
for selector in ("button.login-submit", "button[type='submit']"):
locator = page.locator(selector)
if locator.count() > 0:
locator.first.click(timeout=10_000)
return
candidates = [
page.get_by_role("menuitem", name=target, exact=True),
page.get_by_role("button", name=target, exact=True),
page.get_by_role("link", name=target, exact=True),
page.get_by_text(target, exact=True),
page.get_by_text(target, exact=False),
]
for locator in candidates:
if locator.count() > 0:
locator.first.click(timeout=10_000)
return
raise RuntimeError(f"未找到可点击元素:{target}")
def _execute_step(page, base_url: str, step: dict[str, Any]) -> None:
action = str(step.get("action", "")).lower()
if action == "goto":
target = step.get("target") or step.get("url") or "/"
url = resolve_url(base_url, str(target))
page.goto(url, wait_until="networkidle", timeout=60_000)
return
if action == "fill":
field = str(step.get("field") or step.get("target") or "账号")
_fill_field(page, field, step)
return
if action == "click":
target = step.get("target") or step.get("text")
if not target:
raise RuntimeError("click 步骤缺少 target")
_click_target(page, str(target))
page.wait_for_timeout(800)
return
if action == "wait":
timeout = int(step.get("timeout") or 60_000)
if step.get("url"):
_wait_for_url_pattern(page, str(step["url"]), timeout=timeout)
return
if step.get("selector"):
page.locator(interpolate(str(step["selector"]))).wait_for(timeout=30_000)
return
if step.get("text"):
page.get_by_text(interpolate(str(step["text"])), exact=False).wait_for(timeout=30_000)
return
ms = int(step.get("ms") or 1500)
page.wait_for_timeout(ms)
return
if action == "press":
key = str(step.get("key") or step.get("target") or "Enter")
page.keyboard.press(key)
return
if action == "screenshot":
return
raise RuntimeError(f"未知步骤 action={action}")
def run_browser_scenario_sync(scenario: BrowserScenario) -> BrowserResult:
from playwright.sync_api import sync_playwright
stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
slug = (scenario.name or "browser").replace(" ", "-")
output = SCREENSHOT_DIR / f"{slug}-{stamp}.png"
output.parent.mkdir(parents=True, exist_ok=True)
step_log: list[str] = []
final_url = scenario.base_url
with sync_playwright() as playwright:
browser = playwright.chromium.launch(headless=True)
page = browser.new_page(viewport={"width": 1280, "height": 720})
steps = list(scenario.steps)
if steps and steps[-1].get("action") != "screenshot" and not any(
s.get("action") == "screenshot" for s in steps
):
steps.append({"action": "screenshot"})
for index, step in enumerate(steps):
label = _step_label(step, index)
logger.info("执行步骤 %s", label)
action = str(step.get("action", "")).lower()
if action == "screenshot":
page.wait_for_timeout(int(step.get("ms") or 1500))
page.screenshot(path=str(output), full_page=False, type="png")
final_url = page.url
step_log.append(label + "")
continue
try:
_execute_step(page, scenario.base_url, step)
final_url = page.url
step_log.append(label + "")
except Exception as exc:
err = _page_error_text(page)
detail = f"{err}" if err else ""
raise RuntimeError(f"步骤失败:{label} @ {page.url}{detail}") from exc
browser.close()
return BrowserResult(
scenario_name=scenario.name,
base_url=scenario.base_url,
final_url=final_url,
screenshot_path=output,
step_count=len(steps),
step_log=step_log,
)

View File

@@ -1,26 +0,0 @@
"""浏览器自动化步骤模型。"""
from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
@dataclass
class BrowserScenario:
name: str | None
base_url: str
steps: list[dict[str, Any]]
source: str = "natural"
@dataclass
class BrowserResult:
scenario_name: str | None
base_url: str
final_url: str
screenshot_path: Path
step_count: int
started_dev_server: bool = False
step_log: list[str] = field(default_factory=list)

View File

@@ -1,307 +0,0 @@
"""解析自然语言 / YAML / 场景名 → 浏览器步骤。"""
from __future__ import annotations
import re
from pathlib import Path
from typing import Any
import yaml
import env_config
from browser_env import default_base_url, interpolate
from browser_models import BrowserScenario
SCENARIO_DIRS = [
Path(__file__).resolve().parent / "scenarios",
Path(__file__).resolve().parent.parent / "scenarios",
]
def _project_cwd() -> Path:
raw = env_config.env("CURSOR_CWD", r"d:\LY\test\tech") or r"d:\LY\test\tech"
return Path(raw).resolve()
def _strip_mention(text: str) -> str:
return re.sub(r"@\S+\s*", "", text).strip()
def _scenario_search_dirs() -> list[Path]:
dirs = list(SCENARIO_DIRS)
dirs.append(_project_cwd() / ".browser-scenarios")
custom = (env_config.env("BROWSER_SCENARIOS_DIR") or "").strip()
if custom:
dirs.append(Path(custom).resolve())
return dirs
def is_browser_intent(text: str) -> bool:
raw = _strip_mention(text)
if not raw:
return False
if re.match(r"^(browser|网页|网页操作|操作)\b", raw, re.IGNORECASE):
return True
if re.search(r"```(?:yaml|yml)", raw, re.IGNORECASE):
return True
if re.search(r"(?m)^browser\s*:", raw, re.IGNORECASE):
return True
if re.match(r"^(preview|截图|预览|截屏)\s", raw, re.IGNORECASE):
if not re.search(r"[,。;;]|然后|输入|点击|填写|访问|打开|登录", raw):
return False
if len(_split_segments(text)) >= 2:
return True
verbs = 0
for pattern in (r"访问", r"打开", r"输入", r"填写", r"点击", r"点选", r"选择", r"登录"):
if re.search(pattern, raw):
verbs += 1
return verbs >= 2
def _load_yaml_scenario(path: Path) -> BrowserScenario:
data = yaml.safe_load(path.read_text(encoding="utf-8"))
if not isinstance(data, dict):
raise RuntimeError(f"场景文件格式错误:{path}")
base_url = interpolate(str(data.get("base_url") or default_base_url()))
steps = data.get("steps")
if not isinstance(steps, list) or not steps:
raise RuntimeError(f"场景缺少 steps{path}")
return BrowserScenario(
name=data.get("name") or path.stem,
base_url=base_url,
steps=_normalize_steps(steps),
source=f"file:{path.name}",
)
def _find_scenario_file(name: str) -> Path | None:
slug = name.strip().replace(" ", "-")
for directory in _scenario_search_dirs():
for candidate in (directory / f"{slug}.yaml", directory / f"{slug}.yml"):
if candidate.exists():
return candidate
return None
def _normalize_steps(raw_steps: list[Any]) -> list[dict[str, Any]]:
normalized: list[dict[str, Any]] = []
for item in raw_steps:
if isinstance(item, str):
normalized.append({"action": item})
continue
if not isinstance(item, dict) or not item:
raise RuntimeError(f"无效步骤:{item!r}")
if "action" in item:
normalized.append(dict(item))
continue
if len(item) == 1:
action, payload = next(iter(item.items()))
step = {"action": action}
if payload is not None:
if isinstance(payload, dict):
step.update(payload)
elif action == "wait" and isinstance(payload, int):
step["ms"] = payload
elif action == "wait" and isinstance(payload, str) and payload.isdigit():
step["ms"] = int(payload)
else:
step["target"] = payload
normalized.append(step)
continue
raise RuntimeError(f"无效步骤:{item!r}")
return normalized
def _parse_inline_dsl(text: str) -> BrowserScenario | None:
raw = _strip_mention(text)
match = re.search(r"(?ms)^browser\s*:\s*\n(.+)$", raw, re.IGNORECASE)
if not match:
return None
steps: list[dict[str, Any]] = []
for line in match.group(1).splitlines():
line = line.strip()
if not line or line.startswith("#"):
continue
line = re.sub(r"^[-*]\s*", "", line)
if not line:
continue
steps.append(_parse_dsl_line(line))
if not steps:
return None
return BrowserScenario(
name="inline",
base_url=default_base_url(),
steps=steps,
source="inline-dsl",
)
def _parse_dsl_line(line: str) -> dict[str, Any]:
parts = line.split(None, 2)
action = parts[0].lower()
if action == "goto":
return {"action": "goto", "target": parts[1] if len(parts) > 1 else "/"}
if action == "click":
return {"action": "click", "target": " ".join(parts[1:])}
if action == "fill":
if len(parts) < 3:
raise RuntimeError(f"fill 语法fill 字段 值({line}")
return {"action": "fill", "field": parts[1], "value": parts[2]}
if action == "wait":
payload = parts[1] if len(parts) > 1 else "1500"
if payload.isdigit():
return {"action": "wait", "ms": int(payload)}
return {"action": "wait", "url": payload}
if action in {"screenshot", "shot"}:
return {"action": "screenshot"}
raise RuntimeError(f"未知 DSL 步骤:{line}")
def _split_segments(text: str) -> list[str]:
raw = _strip_mention(text)
raw = re.sub(r"^(browser|网页|网页操作|操作)\s*[:]?\s*", "", raw, flags=re.IGNORECASE)
raw = re.sub(r"然后截图|再截图|最后截图", "截图", raw)
chunks = re.split(r"[,。;;]\s*|\s+然后\s+|\s+接着\s+|\s+并\s*", raw)
expanded: list[str] = []
for chunk in chunks:
chunk = chunk.strip()
if not chunk:
continue
subchunks = re.split(r"\s+然后\s+", chunk)
if "" in chunk and len(subchunks) == 1:
subchunks = re.split(r"(?<=[登录页表单])后(?=[进入打开等待点击访问])", chunk)
for part in subchunks:
part = part.strip()
if part:
expanded.append(part)
return expanded
def _parse_segment(segment: str) -> list[dict[str, Any]]:
seg = segment.strip()
if not seg or seg.lower() in {"browser", "网页操作"}:
return []
if re.fullmatch(r"截图|截屏", seg, re.IGNORECASE):
return [{"action": "screenshot"}]
match = re.search(r"访问登录页|打开登录页|进入登录页|运行登录页", seg, re.IGNORECASE)
if match:
return [{"action": "goto", "target": "/login"}]
match = re.search(r"输入账号密码|填写账号密码|输入账号和密码", seg, re.IGNORECASE)
if match:
return [
{"action": "fill", "field": "账号", "value": "{{PREVIEW_LOGIN_USER}}"},
{"action": "fill", "field": "密码", "value": "{{PREVIEW_LOGIN_PASSWORD}}"},
]
match = re.search(r"输入账号|填写账号|输入用户名|填写用户名", seg, re.IGNORECASE)
if match:
return [{"action": "fill", "field": "账号", "value": "{{PREVIEW_LOGIN_USER}}"}]
match = re.search(r"输入密码|填写密码", seg, re.IGNORECASE)
if match:
return [{"action": "fill", "field": "密码", "value": "{{PREVIEW_LOGIN_PASSWORD}}"}]
match = re.search(r"进入主页|进入首页|打开主页|打开首页|等待主页", seg, re.IGNORECASE)
if match:
return [{"action": "wait", "url": "**/app/**"}]
match = re.search(r"等待\s*(\d+)\s*秒", seg, re.IGNORECASE)
if match:
return [{"action": "wait", "ms": int(match.group(1)) * 1000}]
match = re.search(
r"(?:点击|点选|选择)\s*(.+?)(?:菜单|按钮|链接)?$",
seg,
re.IGNORECASE,
)
if match:
target = match.group(1).strip()
target = re.sub(r"(然后|再|并)?\s*(截图|截屏).*$", "", target, flags=re.IGNORECASE).strip()
target = re.sub(r"(然后|再|之后)$", "", target).strip()
target = re.sub(r"(菜单|按钮|链接)$", "", target).strip()
if target:
return [{"action": "click", "target": target}]
match = re.search(
r"(?:访问|打开|进入)\s*(https?://\S+|/\S+|登录页|主页|首页)",
seg,
re.IGNORECASE,
)
if match:
target = match.group(1)
mapping = {"登录页": "/login", "主页": "/app/dashboard", "首页": "/app/dashboard"}
return [{"action": "goto", "target": mapping.get(target, target)}]
return []
def parse_natural_language(text: str) -> BrowserScenario | None:
segments = _split_segments(text)
steps: list[dict[str, Any]] = []
for segment in segments:
steps.extend(_parse_segment(segment))
if not steps:
return None
if not any(step.get("action") == "screenshot" for step in steps):
if re.search(r"截图|截屏", text, re.IGNORECASE):
steps.append({"action": "screenshot"})
if not steps:
return None
return BrowserScenario(
name="natural",
base_url=default_base_url(),
steps=steps,
source="natural-language",
)
def parse_browser_request(text: str) -> BrowserScenario | None:
if not is_browser_intent(text):
return None
raw = _strip_mention(text)
yaml_block = re.search(r"```(?:yaml|yml)\s*\n(.+?)```", raw, re.IGNORECASE | re.DOTALL)
if yaml_block:
data = yaml.safe_load(yaml_block.group(1))
if isinstance(data, dict):
base_url = interpolate(str(data.get("base_url") or default_base_url()))
steps = data.get("steps") or []
return BrowserScenario(
name=data.get("name") or "yaml-inline",
base_url=base_url,
steps=_normalize_steps(steps),
source="yaml-inline",
)
inline = _parse_inline_dsl(text)
if inline:
return inline
match = re.match(r"^(browser|网页|网页操作|操作)\s+([\w\-./]+)\s*$", raw, re.IGNORECASE)
if match:
path = _find_scenario_file(match.group(2))
if not path:
raise RuntimeError(f"未找到场景文件:{match.group(2)}.yaml")
return _load_yaml_scenario(path)
scenario = parse_natural_language(text)
if scenario:
return scenario
default_name = (env_config.env("BROWSER_DEFAULT_SCENARIO") or "").strip()
if default_name:
path = _find_scenario_file(default_name)
if path:
return _load_yaml_scenario(path)
return None

View File

@@ -1,87 +0,0 @@
"""浏览器自动化服务:解析场景 + 启动 dev server + 执行步骤。"""
from __future__ import annotations
import asyncio
import logging
from urllib.parse import urlparse
from browser_executor import run_browser_scenario_sync
from browser_models import BrowserResult, BrowserScenario
from browser_parser import parse_browser_request
from preview_service import (
_package_dev_script,
_preview_port,
_project_cwd,
_startup_timeout,
_wait_for_port,
)
logger = logging.getLogger(__name__)
def _ensure_dev_server(base_url: str) -> bool:
parsed = urlparse(base_url)
host = parsed.hostname or "127.0.0.1"
port = parsed.port or (443 if parsed.scheme == "https" else 80)
if _wait_for_port(host, port, timeout=3):
return False
cwd = _project_cwd()
dev_command = _package_dev_script(cwd)
if not dev_command:
raise RuntimeError(
f"无法访问 {base_url},且未找到可启动的 dev 脚本。"
"请先手动启动前端,或设置 PREVIEW_URL。"
)
import subprocess
logger.info("启动 dev server: %s (cwd=%s)", dev_command, cwd)
proc = subprocess.Popen(
dev_command,
cwd=str(cwd),
shell=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
)
if not _wait_for_port(host, port, timeout=_startup_timeout()):
err = ""
if proc.stderr:
err = proc.stderr.read().decode("utf-8", errors="replace")[-1000:]
proc.kill()
raise RuntimeError(
f"dev server 在 {_startup_timeout()}s 内未就绪 ({base_url})。"
f"{(' 日志: ' + err) if err else ''}"
)
return True
async def run_browser_automation(text: str) -> BrowserResult:
scenario = parse_browser_request(text)
if scenario is None:
raise RuntimeError("无法解析网页操作步骤")
started = await asyncio.to_thread(_ensure_dev_server, scenario.base_url)
result = await asyncio.to_thread(run_browser_scenario_sync, scenario)
result.started_dev_server = started
return result
def format_browser_caption(result: BrowserResult) -> str:
lines = [
"**网页操作完成**",
f"> 场景:`{result.scenario_name or '自定义'}`",
f"> 起始:`{result.base_url}`",
f"> 最终:`{result.final_url}`",
f"> 步骤数:{result.step_count}",
]
if result.started_dev_server:
lines.append("> dev server已自动启动")
if result.step_log:
lines.append("")
lines.append("执行记录:")
for item in result.step_log[-8:]:
lines.append(f"- {item}")
return "\n".join(lines)

View File

@@ -1,106 +0,0 @@
"""通过 Cursor SDK 执行用户任务。"""
from __future__ import annotations
import asyncio
import logging
import re
from typing import Awaitable, Callable
import env_config
from bridge_manager import warm_cursor_bridge
logger = logging.getLogger(__name__)
_cursor_lock = asyncio.Lock()
WECHAT_SYSTEM_PREFIX = """你是企业微信群里的 Skills 助手,正在回复群成员的消息。
要求:
- 用简洁的中文回答(除非用户用其他语言提问)
- 使用企业微信支持的 Markdown 子集(加粗、链接、列表;避免复杂表格)
- 直接给出结论,不要冗长铺垫
- 若任务涉及 skills.sh可说明安装命令 `npx skills add owner/repo/skill-name`
- **不要**在回复里写 `[图片]` 占位符;企微无法通过 Markdown 显示图片
- 若用户要页面截图,请明确告知其发送:`截图` 或 `preview`(由 bot 自动发图)
用户任务:
"""
def _cursor_settings() -> dict[str, str | int]:
timeout_raw = env_config.env("CURSOR_TIMEOUT", "600") or "600"
return {
"api_key": env_config.env("CURSOR_API_KEY"),
"cwd": env_config.env("CURSOR_CWD", r"d:\LY\test\tech") or r"d:\LY\test\tech",
"model": env_config.env("CURSOR_MODEL", "composer-2.5") or "composer-2.5",
"timeout": int(timeout_raw),
}
def strip_mention(text: str) -> str:
return re.sub(r"@\S+\s*", "", text).strip()
def _build_prompt(task: str) -> str:
return WECHAT_SYSTEM_PREFIX + task.strip()
def execute_cursor_task_sync(task: str) -> str:
from cursor_sdk import Agent, AgentOptions, CursorAgentError, LocalAgentOptions
settings = _cursor_settings()
api_key = settings["api_key"]
if not api_key:
raise RuntimeError(
"未配置 CURSOR_API_KEY。请在 bot/.env 中设置,"
"密钥见 https://cursor.com/dashboard/integrations"
)
warm_cursor_bridge()
cwd = str(settings["cwd"])
prompt = _build_prompt(task)
logger.info("Cursor 执行任务 cwd=%s model=%s", cwd, settings["model"])
try:
result = Agent.prompt(
prompt,
AgentOptions(
api_key=api_key,
model=settings["model"],
local=LocalAgentOptions(cwd=cwd),
),
)
except CursorAgentError as exc:
raise RuntimeError(
f"Cursor 启动失败:{exc.message}"
+ ("(可重试)" if exc.is_retryable else "")
) from exc
if result.status == "error":
detail = result.result or "运行失败,无详细错误"
raise RuntimeError(f"Cursor 执行失败:{detail}")
text = (result.result or "").strip()
if not text:
return "Cursor 已完成任务,但没有返回文本内容。"
return text
async def run_cursor_task(
task: str,
on_progress: Callable[[str], Awaitable[None]] | None = None,
) -> str:
timeout = int(_cursor_settings()["timeout"])
if on_progress:
await on_progress("Cursor 正在执行任务,请稍候…")
async with _cursor_lock:
try:
return await asyncio.wait_for(
asyncio.to_thread(execute_cursor_task_sync, task),
timeout=timeout,
)
except asyncio.TimeoutError as exc:
raise RuntimeError(f"Cursor 执行超时(>{timeout}s") from exc

View File

@@ -1,21 +0,0 @@
"""加载 bot/.env供各模块在 import 时统一读取环境变量。"""
from __future__ import annotations
import os
import sys
from pathlib import Path
from dotenv import load_dotenv
_BOT_DIR = Path(__file__).resolve().parent
_REPO_ROOT = _BOT_DIR.parent
if str(_REPO_ROOT) not in sys.path:
sys.path.insert(0, str(_REPO_ROOT))
load_dotenv(_BOT_DIR / ".env")
load_dotenv(_BOT_DIR / ".env.local", override=True)
def env(key: str, default: str | None = None) -> str | None:
return os.getenv(key, default)

View File

@@ -1,58 +0,0 @@
"""从文本/Cursor 回复中解析本地截图路径。"""
from __future__ import annotations
import re
from pathlib import Path
import env_config
IMAGE_SUFFIXES = (".png", ".jpg", ".jpeg", ".webp")
def _project_cwd() -> Path:
raw = env_config.env("CURSOR_CWD", r"d:\LY\test\tech") or r"d:\LY\test\tech"
return Path(raw).resolve()
def _resolve_candidate(raw: str, cwd: Path) -> Path | None:
cleaned = raw.strip().strip("`\"'[]()")
if not cleaned or cleaned.startswith("http"):
return None
path = Path(cleaned)
if not path.is_absolute():
path = cwd / path
try:
resolved = path.resolve()
except OSError:
return None
if resolved.is_file() and resolved.suffix.lower() in IMAGE_SUFFIXES:
return resolved
return None
def find_image_paths(text: str) -> list[Path]:
cwd = _project_cwd()
seen: set[Path] = set()
found: list[Path] = []
patterns = [
r"(?:保存(?:至|到)|saved\s+to|screenshot\s*[:])\s*([^\s\n\]]+\.(?:png|jpe?g|webp))",
r"([A-Za-z]:\\[^\s\n\]]+\.(?:png|jpe?g|webp))",
r"([^\s\n\]]+\.(?:png|jpe?g|webp))",
]
for pattern in patterns:
for match in re.finditer(pattern, text, re.IGNORECASE):
path = _resolve_candidate(match.group(1), cwd)
if path and path not in seen:
seen.add(path)
found.append(path)
return found
def strip_fake_image_markdown(text: str) -> str:
text = re.sub(r"^\s*\[图片\]\s*$", "", text, flags=re.MULTILINE)
text = re.sub(r"\n{3,}", "\n\n", text)
return text.strip()

View File

@@ -1,156 +0,0 @@
"""企业微信智能机器人 · Skills 助手skills 快查 + 截图预览 + Cursor 执行任务)。"""
from __future__ import annotations
import logging
import sys
import env_config
from bridge_manager import shutdown_cursor_bridge, warm_cursor_bridge
from router import route_message, routing_mode
from skills_service import handle_command, warm_feed_cache
from wecom_media import reply_image, upload_image
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s [%(levelname)s] %(message)s",
)
logger = logging.getLogger("skills-bot")
BOT_ID = env_config.env("WECOM_BOT_ID") or env_config.env("WECHAT_BOT_ID")
BOT_SECRET = env_config.env("WECOM_BOT_SECRET") or env_config.env("WECHAT_BOT_SECRET")
def _require_credentials() -> None:
if not BOT_ID or not BOT_SECRET:
print(
"请设置环境变量 WECOM_BOT_ID 和 WECOM_BOT_SECRET\n"
"(企业微信 → 智能机器人 → API 模式 → 长连接)",
file=sys.stderr,
)
sys.exit(1)
def create_client():
from aibot import WSClient, WSClientOptions, generate_req_id
ws_client = WSClient(
WSClientOptions(
bot_id=BOT_ID,
secret=BOT_SECRET,
)
)
@ws_client.on("authenticated")
def on_authenticated():
logger.info("企业微信长连接认证成功,路由模式=%s", routing_mode())
cursor_key = env_config.env("CURSOR_API_KEY")
if cursor_key:
logger.info("CURSOR_API_KEY 已加载(%s…)", cursor_key[:8])
try:
warm_cursor_bridge()
logger.info("Cursor bridge 预启动完成")
except Exception as exc:
logger.warning("Cursor bridge 预启动失败Cursor 任务时会重试): %s", exc)
else:
logger.warning("CURSOR_API_KEY 未配置Cursor 任务将失败")
try:
warm_feed_cache()
logger.info("skills 数据预加载完成")
except Exception as exc:
logger.warning("skills 数据预加载失败: %s", exc)
@ws_client.on("event.enter_chat")
async def on_enter_chat(frame):
help_text = handle_command("help")
extra = (
"\n\n---\n"
"**单页截图**`preview` / `截图` / `预览 [路径或URL]`\n"
"**网页操作**:自然语言多步操作,或 `browser 场景名`\n"
"例:`访问登录页,输入账号密码,点击登录,点击智能体管理,截图`\n"
"场景文件:`bot/scenarios/*.yaml`(可用 `browser xiaobao-agent-manage`"
)
await ws_client.reply_welcome(
frame,
{
"msgtype": "markdown",
"markdown": {"content": help_text + extra},
},
)
@ws_client.on("message.text")
async def on_text(frame):
body = frame.get("body", {})
content = body.get("text", {}).get("content", "")
logger.info("收到消息: %s", content)
stream_id = generate_req_id("stream")
last_progress = ""
async def on_progress(message: str) -> None:
nonlocal last_progress
if message != last_progress:
last_progress = message
await ws_client.reply_stream(frame, stream_id, message, False)
await ws_client.reply_stream(frame, stream_id, "收到,正在处理…", False)
try:
result = await route_message(content, on_progress=on_progress)
reply = result.text
logger.info(
"回复来源: %s, 文本长度=%d, 图片=%s",
result.source,
len(reply),
result.image_path or "-",
)
except Exception as exc:
logger.exception("处理失败")
reply = f"处理失败:{exc}"
result = None
if len(reply) > 3800:
reply = reply[:3800] + "\n\n> …内容已截断"
await ws_client.reply_stream(frame, stream_id, reply, True)
if result and result.image_path:
try:
media_id = await upload_image(ws_client, result.image_path)
await reply_image(ws_client, frame, media_id)
logger.info("图片已发送到企微: %s", result.image_path)
except Exception as exc:
logger.exception("发送图片失败")
await ws_client.reply(
frame,
{
"msgtype": "markdown",
"markdown": {
"content": f"截图文件:`{result.image_path}`\n发图失败:{exc}\n\n请确认 bot 已重启,或发送 `截图` 重试。",
},
},
)
@ws_client.on("error")
def on_error(error):
logger.error("连接错误: %s", error)
@ws_client.on("disconnected")
def on_disconnected(reason):
logger.warning("连接断开: %s", reason)
return ws_client
def main() -> None:
import atexit
atexit.register(shutdown_cursor_bridge)
_require_credentials()
client = create_client()
logger.info("启动 Skills 助手Bot ID=%s", BOT_ID[:8] if BOT_ID else "?")
client.run()
if __name__ == "__main__":
main()

View File

@@ -1,255 +0,0 @@
"""在 CURSOR_CWD 启动/访问前端并截图(单页,不含多步操作)。"""
from __future__ import annotations
import asyncio
import json
import logging
import re
import socket
import subprocess
import time
from dataclasses import dataclass
from datetime import datetime
from pathlib import Path
from urllib.parse import urlparse
import env_config
logger = logging.getLogger(__name__)
SCREENSHOT_DIR = Path(__file__).resolve().parent / ".cache" / "screenshots"
@dataclass
class PreviewResult:
url: str
screenshot_path: Path
started_dev_server: bool
final_url: str | None = None
@dataclass
class PreviewRequest:
url: str | None
port: int | None
def _project_cwd() -> Path:
raw = env_config.env("CURSOR_CWD", r"d:\LY\test\tech") or r"d:\LY\test\tech"
return Path(raw).resolve()
def _preview_port() -> int:
raw = env_config.env("PREVIEW_PORT", "5173") or "5173"
return int(raw)
def _startup_timeout() -> int:
raw = env_config.env("PREVIEW_STARTUP_TIMEOUT", "120") or "120"
return int(raw)
def _dev_command() -> str:
return env_config.env("PREVIEW_DEV_COMMAND", "npm run dev") or "npm run dev"
def parse_preview_command(text: str) -> tuple[str | None, int | None] | None:
raw = re.sub(r"@\S+\s*", "", text).strip()
if not raw:
return None
m = re.match(
r"^(preview|截图|预览|截屏)(?:\s+(https?://\S+|/\S*))?(?:\s+(\d{2,5}))?$",
raw,
re.IGNORECASE,
)
if not m:
return None
url_part = m.group(2)
port_part = m.group(3)
port = int(port_part) if port_part else None
if url_part and url_part.startswith("/"):
port = port or _preview_port()
return f"http://127.0.0.1:{port}{url_part}", port
return url_part, port
def resolve_preview_request(text: str) -> PreviewRequest | None:
explicit = parse_preview_command(text)
if explicit is not None:
url_override, port_override = explicit
return PreviewRequest(url=url_override, port=port_override)
if not is_preview_intent(text):
return None
url_override = extract_url_from_text(text)
if not url_override:
env_url = env_config.env("PREVIEW_URL")
url_override = env_url.strip() if env_url else f"http://127.0.0.1:{_preview_port()}/"
return PreviewRequest(url=url_override, port=None)
_PREVIEW_INTENT = re.compile(
r"^(preview|截图|预览|截屏)\b|"
r"(页面预览|运行.*(前端|项目|页面)|"
r"打开.*(前端|页面|项目)|"
r"访问.*(并)?.*(截图|截屏)|"
r"启动.*(前端|项目|dev|服务).*(截图|截屏)?)",
re.IGNORECASE,
)
def is_preview_intent(text: str) -> bool:
raw = re.sub(r"@\S+\s*", "", text).strip()
if parse_preview_command(text) is not None:
return True
return bool(_PREVIEW_INTENT.search(raw))
def extract_url_from_text(text: str) -> str | None:
raw = re.sub(r"@\S+\s*", "", text)
match = re.search(
r"(https?://[^\s\]`\"']+|localhost:\d+[/\w\-./]*)",
raw,
re.IGNORECASE,
)
if not match:
return None
url = match.group(1).rstrip(".,,。")
if url.lower().startswith("localhost"):
url = "http://" + url
return url
def _capture_screenshot_sync(url: str, output: Path) -> str:
from playwright.sync_api import sync_playwright
output.parent.mkdir(parents=True, exist_ok=True)
with sync_playwright() as p:
browser = p.chromium.launch(headless=True)
page = browser.new_page(viewport={"width": 1280, "height": 720})
page.goto(url, wait_until="networkidle", timeout=60_000)
page.wait_for_timeout(1500)
page.screenshot(path=str(output), full_page=False, type="png")
final_url = page.url
browser.close()
return final_url
def _wait_for_port(host: str, port: int, timeout: int) -> bool:
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
try:
with socket.create_connection((host, port), timeout=2):
return True
except OSError:
time.sleep(1)
return False
def _resolve_target_url(url_override: str | None, port_override: int | None) -> tuple[str, str | None]:
if url_override:
parsed = urlparse(url_override)
if parsed.scheme and parsed.netloc:
return url_override, None
raise RuntimeError(f"无效 URL{url_override}")
env_url = env_config.env("PREVIEW_URL")
if env_url:
return env_url.strip(), None
port = port_override or _preview_port()
cwd = _project_cwd()
dev_script = _package_dev_script(cwd)
base = f"http://127.0.0.1:{port}/"
return base, dev_script
def _package_dev_script(cwd: Path) -> str | None:
pkg = cwd / "package.json"
if not pkg.exists():
return None
try:
data = json.loads(pkg.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return None
scripts = data.get("scripts") or {}
for key in ("dev", "preview", "start"):
if scripts.get(key):
cmd = _dev_command()
if key != "dev" and cmd == "npm run dev":
return f"npm run {key}"
return cmd
return None
def _capture_preview_sync(url: str, dev_command: str | None) -> PreviewResult:
cwd = _project_cwd()
parsed = urlparse(url)
host = parsed.hostname or "127.0.0.1"
port = parsed.port or (443 if parsed.scheme == "https" else 80)
dev_proc: subprocess.Popen | None = None
started = False
if dev_command:
if _wait_for_port(host, port, timeout=3):
logger.info("检测到端口 %s 已监听,跳过启动 dev server", port)
else:
logger.info("启动 dev server: %s (cwd=%s)", dev_command, cwd)
dev_proc = subprocess.Popen(
dev_command,
cwd=str(cwd),
shell=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
)
started = True
if not _wait_for_port(host, port, timeout=_startup_timeout()):
err = ""
if dev_proc.stderr:
err = dev_proc.stderr.read().decode("utf-8", errors="replace")[-1000:]
raise RuntimeError(
f"dev server 在 {_startup_timeout()}s 内未就绪 ({url})。"
f"{(' 日志: ' + err) if err else ''}"
)
else:
if not _wait_for_port(host, port, timeout=5):
raise RuntimeError(
f"无法访问 {url}。请在 CURSOR_CWD 放置前端项目,"
"或先手动启动 dev server或设置 PREVIEW_URL。"
)
stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
output = SCREENSHOT_DIR / f"preview-{stamp}.png"
try:
final_url = _capture_screenshot_sync(url, output)
finally:
if dev_proc and dev_proc.poll() is None:
dev_proc.terminate()
try:
dev_proc.wait(timeout=5)
except subprocess.TimeoutExpired:
dev_proc.kill()
return PreviewResult(
url=url,
screenshot_path=output,
started_dev_server=started,
final_url=final_url,
)
async def capture_preview(
url_override: str | None = None,
port_override: int | None = None,
) -> PreviewResult:
url, dev_command = _resolve_target_url(url_override, port_override)
return await asyncio.to_thread(_capture_preview_sync, url, dev_command)

View File

@@ -1,7 +0,0 @@
wecom-aibot-python-sdk>=1.0.2
python-dotenv>=1.0.0
httpx>=0.27.0
certifi>=2024.0.0
cursor-sdk>=0.1.0
playwright>=1.49.0
PyYAML>=6.0.0

View File

@@ -1,109 +0,0 @@
"""消息路由skills 快查 / 网页操作 / 截图预览 / Cursor 通用任务。"""
from __future__ import annotations
import re
import env_config
from browser_parser import is_browser_intent, parse_browser_request
from browser_service import format_browser_caption, run_browser_automation
from cursor_runner import run_cursor_task, strip_mention
from image_extract import find_image_paths, strip_fake_image_markdown
from preview_service import capture_preview, is_preview_intent, resolve_preview_request
from skills_service import handle_command, parse_command
from bot_types import RouteResult
def routing_mode() -> str:
return (env_config.env("ROUTING_MODE", "hybrid") or "hybrid").lower()
def _normalize(text: str) -> str:
return re.sub(r"@\S+\s*", "", text).strip().lower()
def is_skills_fast_command(text: str) -> bool:
raw = _normalize(text)
if not raw:
return True
if raw in {"help", "帮助", "?", "h"}:
return True
cmd = parse_command(text)
if cmd.kind in {"help", "list", "detail"}:
return True
if cmd.kind == "search" and re.match(r"^(search|搜索|find|查)\s+", raw):
return True
return False
async def _run_browser(text: str, on_progress=None) -> RouteResult:
if parse_browser_request(text) is None:
raise RuntimeError("无法解析网页操作步骤")
if on_progress:
await on_progress("正在按步骤执行网页操作…")
result = await run_browser_automation(text)
return RouteResult(
source="browser",
text=format_browser_caption(result),
image_path=str(result.screenshot_path),
)
async def _run_preview(text: str, on_progress=None) -> RouteResult:
preview_req = resolve_preview_request(text)
if preview_req is None:
raise RuntimeError("无法解析截图请求")
if on_progress:
await on_progress(f"正在访问并截图:{preview_req.url or '默认地址'}")
result = await capture_preview(preview_req.url, preview_req.port)
caption = (
f"**页面预览**\n"
f"> URL`{result.final_url or result.url}`\n"
f"> 项目:`{env_config.env('CURSOR_CWD', '')}`\n"
f"> dev server{'已自动启动' if result.started_dev_server else '使用已有服务'}"
)
return RouteResult(
source="preview",
text=caption,
image_path=str(result.screenshot_path),
)
async def route_message(text: str, on_progress=None) -> RouteResult:
task = strip_mention(text)
if not task:
return RouteResult("skills", handle_command("help"))
if is_browser_intent(text):
return await _run_browser(text, on_progress=on_progress)
if resolve_preview_request(text) is not None:
return await _run_preview(text, on_progress=on_progress)
mode = routing_mode()
if mode == "skills":
return RouteResult("skills", handle_command(text))
if mode == "cursor" or not is_skills_fast_command(text):
reply = await run_cursor_task(task, on_progress=on_progress)
reply = strip_fake_image_markdown(reply)
image_path: str | None = None
paths = find_image_paths(reply)
if paths:
image_path = str(paths[0])
elif is_preview_intent(text) or is_browser_intent(text):
if on_progress:
await on_progress("未找到截图文件,改用 Playwright 自动执行…")
if is_browser_intent(text):
return await _run_browser(text, on_progress=on_progress)
return await _run_preview(text, on_progress=on_progress)
return RouteResult("cursor", reply, image_path=image_path)
return RouteResult("skills", handle_command(text))

View File

@@ -1,17 +0,0 @@
name: xiaobao-agent-manage
description: 登录后打开智能体管理并截图
steps:
- goto: /login
- fill:
field: 账号
value: "{{PREVIEW_LOGIN_USER}}"
- fill:
field: 密码
value: "{{PREVIEW_LOGIN_PASSWORD}}"
- click: 登录
- wait:
url: "**/app/**"
timeout: 60000
- click: 智能体管理
- wait: 1500
- screenshot

View File

@@ -1,226 +0,0 @@
"""skills.sh 快查命令解析与企微回复格式化。"""
from __future__ import annotations
import re
import sys
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Literal
_REPO_ROOT = Path(__file__).resolve().parent.parent
if str(_REPO_ROOT) not in sys.path:
sys.path.insert(0, str(_REPO_ROOT))
from shared.skills_data import Board, board_items, format_installs, load_feed, warm_feed_cache
__all__ = ["Command", "handle_command", "parse_command", "warm_feed_cache"]
@dataclass
class Command:
kind: Literal["help", "list", "search", "detail"]
board: Board = "trending"
limit: int = 10
query: str = ""
def _normalize_text(text: str) -> str:
text = re.sub(r"@\S+\s*", "", text)
return text.strip().lower()
def _parse_limit(raw: str | None, default: int = 10) -> int:
if not raw:
return default
try:
n = int(raw)
except ValueError:
return default
return max(1, min(n, 30))
def _match_list(raw: str, board: Board, aliases: str) -> Command | None:
m = re.match(rf"^({aliases})(?:\s+top)?\s*(\d+)?$", raw)
if m:
return Command(kind="list", board=board, limit=_parse_limit(m.group(2)))
m = re.match(rf"^(查|查询)\s+({aliases})(?:\s+top)?\s*(\d+)?$", raw)
if m:
return Command(kind="list", board=board, limit=_parse_limit(m.group(3)))
return None
def parse_command(text: str) -> Command:
raw = _normalize_text(text)
if not raw or raw in {"help", "帮助", "?", "h"}:
return Command(kind="help")
for board, aliases in (
("trending", "trending|趋势|top"),
("hot", "hot|实时|热门"),
("all", "all|总榜|alltime|all-time"),
):
cmd = _match_list(raw, board, aliases)
if cmd:
return cmd
m = re.match(r"^(search|搜索|find|查)\s+(.+)$", raw)
if m:
return Command(kind="search", query=m.group(2).strip(), limit=5)
m = re.match(r"^(detail|详情|skill|info)\s+(.+)$", raw)
if m:
return Command(kind="detail", query=m.group(2).strip())
if raw.startswith("trending") or raw.startswith("趋势"):
parts = raw.split(maxsplit=1)
return Command(kind="list", board="trending", limit=_parse_limit(parts[1] if len(parts) > 1 else None))
return Command(kind="search", query=raw, limit=5)
def _board_title(board: Board) -> str:
return {
"trending": "Trending近期增长",
"hot": "Hot实时热度",
"all": "All Time总安装榜",
}[board]
def format_list(board: Board, limit: int) -> str:
feed = load_feed()
items = board_items(feed, board)[:limit]
updated = feed.get("updatedAt", "未知")[:10]
lines = [
f"**skills.sh {_board_title(board)} Top {limit}**",
f"> 数据更新:{updated}",
"",
]
for i, item in enumerate(items, 1):
title = item.get("title", "?")
source = item.get("source", "?")
installs = format_installs(item.get("installs", 0))
desc = item.get("description", "")
if len(desc) > 80:
desc = desc[:77] + "..."
link = item.get("link", "")
lines.append(f"{i}. **{title}** · {installs}")
lines.append(f" `{source}`")
if desc:
lines.append(f" {desc}")
if link:
lines.append(f" [查看]({link})")
lines.append("")
return "\n".join(lines).strip()
def format_search(query: str, limit: int) -> str:
feed = load_feed()
q = query.lower()
seen: set[str] = set()
matches: list[dict[str, Any]] = []
for board in ("topTrending", "topHot", "topAllTime"):
for item in feed.get(board, []):
item_id = item.get("id") or item.get("title", "")
if item_id in seen:
continue
haystack = " ".join(
[
item.get("title", ""),
item.get("source", ""),
item.get("description", ""),
]
).lower()
if q in haystack:
seen.add(item_id)
matches.append(item)
if len(matches) >= limit:
break
if len(matches) >= limit:
break
if not matches:
return f"未找到与 **{query}** 相关的 skill。\n\n试试:`trending 10` / `hot 10` / `搜索 react`"
lines = [f"**搜索「{query}」** 共 {len(matches)}", ""]
for i, item in enumerate(matches, 1):
title = item.get("title", "?")
source = item.get("source", "?")
installs = format_installs(item.get("installs", 0))
link = item.get("link", "")
lines.append(f"{i}. **{title}** · {installs} · `{source}`")
if link:
lines.append(f" [查看]({link})")
return "\n".join(lines)
def format_detail(name: str) -> str:
feed = load_feed()
q = name.lower().strip()
best: dict[str, Any] | None = None
for board in ("topTrending", "topHot", "topAllTime"):
for item in feed.get(board, []):
title = (item.get("title") or "").lower()
item_id = (item.get("id") or "").lower()
if title == q or q in title or q in item_id:
if best is None or item.get("installs", 0) > best.get("installs", 0):
best = item
if not best:
return f"未找到 skill**{name}**\n\n试试:`搜索 {name}`"
desc = best.get("description", "无描述")
return "\n".join(
[
f"**{best.get('title', '?')}**",
f"`{best.get('source', '?')}`",
f"安装量:**{format_installs(best.get('installs', 0))}**",
"",
desc,
"",
f"[skills.sh 详情]({best.get('link', 'https://skills.sh')})",
"",
f"安装:`npx skills add {best.get('source', '')}/{best.get('title', '')}`",
]
)
def format_help() -> str:
return "\n".join(
[
"**Skills 助手 · 命令帮助**",
"",
"`trending 10` / `趋势 10` — 近期增长榜",
"`hot 10` / `实时 10` — 实时热度榜",
"`all 10` / `总榜 10` — 历史总安装榜",
"`搜索 react` / `search tdd` — 关键词搜索",
"`详情 find-skills` — 查看单个 skill",
"`preview` / `截图` / `预览` — 单页截图",
"`browser 场景名` — 执行 YAML 场景(见 bot/scenarios/",
"自然语言 — 如:访问登录页,输入账号密码,点击登录,点击智能体管理,截图",
"`preview /about 5173` — 指定路径和端口",
"",
"示例:",
"• trending top10",
"• 查 grill",
"• 详情 remotion-render",
]
)
def handle_command(text: str) -> str:
cmd = parse_command(text)
if cmd.kind == "help":
return format_help()
if cmd.kind == "list":
return format_list(cmd.board, cmd.limit)
if cmd.kind == "search":
return format_search(cmd.query, cmd.limit)
if cmd.kind == "detail":
return format_detail(cmd.query)
return format_help()

View File

@@ -1,96 +0,0 @@
"""企业微信 API 模式:上传图片并回复。"""
from __future__ import annotations
import base64
import hashlib
import logging
from pathlib import Path
from typing import Any
from aibot import generate_req_id
logger = logging.getLogger(__name__)
CHUNK_SIZE = 512 * 1024
MAX_IMAGE_BYTES = 9 * 1024 * 1024
def _ensure_image_size(path: Path) -> bytes:
data = path.read_bytes()
if len(data) > MAX_IMAGE_BYTES:
raise RuntimeError(
f"截图过大({len(data) // 1024}KB请缩小页面或使用 viewport 截图(上限 9MB"
)
return data
def _response_body(frame: dict[str, Any]) -> dict[str, Any]:
if frame.get("errcode", 0) != 0:
raise RuntimeError(
f"企微接口错误 errcode={frame.get('errcode')} errmsg={frame.get('errmsg')}"
)
body = frame.get("body")
return body if isinstance(body, dict) else {}
async def upload_image(ws_client: Any, image_path: str | Path) -> str:
path = Path(image_path)
if not path.exists():
raise RuntimeError(f"截图不存在: {path}")
data = _ensure_image_size(path)
md5 = hashlib.md5(data).hexdigest()
chunks = [data[i : i + CHUNK_SIZE] for i in range(0, len(data), CHUNK_SIZE)]
total_chunks = len(chunks)
manager = ws_client._ws_manager
init_frame = await manager.send_reply(
generate_req_id("upload_init"),
{
"type": "image",
"filename": path.name,
"total_size": len(data),
"total_chunks": total_chunks,
"md5": md5,
},
"aibot_upload_media_init",
)
upload_id = _response_body(init_frame).get("upload_id")
if not upload_id:
raise RuntimeError("上传初始化失败:未返回 upload_id")
for index, chunk in enumerate(chunks):
chunk_frame = await manager.send_reply(
generate_req_id("upload_chunk"),
{
"upload_id": upload_id,
"chunk_index": index,
"base64_data": base64.b64encode(chunk).decode("ascii"),
},
"aibot_upload_media_chunk",
)
_response_body(chunk_frame)
finish_frame = await manager.send_reply(
generate_req_id("upload_finish"),
{"upload_id": upload_id},
"aibot_upload_media_finish",
)
media_id = _response_body(finish_frame).get("media_id")
if not media_id:
raise RuntimeError("上传完成但未返回 media_id")
logger.info("图片已上传 media_id=%s", str(media_id)[:12])
return str(media_id)
async def reply_image(ws_client: Any, frame: dict[str, Any], media_id: str) -> None:
await ws_client.reply(
frame,
{
"msgtype": "image",
"image": {"media_id": media_id},
},
)

View File

@@ -1,166 +0,0 @@
version: 1
intl:
categories:
- id: official
name: 厂商官方
icon: 🏢
feeds:
- name: Anthropic Claude 更新
url: https://docs.anthropic.com/en/release-notes/feed
- name: OpenAI
url: https://openai.com/news/rss.xml
- name: Google AI
url: https://blog.google/technology/ai/rss/
- name: DeepMind
url: https://deepmind.google/blog/rss.xml
- name: Meta Engineering
url: https://engineering.fb.com/feed/
- name: Microsoft Research
url: https://www.microsoft.com/en-us/research/feed/
- name: Microsoft Blog
url: https://blogs.microsoft.com/feed/
- name: Cohere
url: https://cohere.com/blog/rss.xml
- name: Cursor Changelog
url: https://cursor.com/changelog/rss.xml
- id: developer
name: Agent / LLM 开发者
icon: 🛠
feeds:
- name: LangChain
url: https://blog.langchain.dev/rss/
- name: Hugging Face
url: https://huggingface.co/blog/feed.xml
- name: Vercel Changelog
url: https://vercel.com/changelog/rss.xml
- name: GitHub Copilot
url: https://github.blog/changelog/label/copilot/feed/
- id: media
name: 综合科技媒体
icon: 📰
feeds:
- name: The Verge AI
url: https://www.theverge.com/rss/ai-artificial-intelligence/index.xml
- name: TechCrunch AI
url: https://techcrunch.com/category/artificial-intelligence/feed/
- name: Ars Technica AI
url: https://arstechnica.com/ai/feed/
- name: Wired AI
url: https://www.wired.com/feed/tag/ai/latest/rss
- name: MIT Tech Review
url: https://www.technologyreview.com/feed/
- name: VentureBeat AI
url: https://venturebeat.com/category/ai/feed/
- id: newsletter
name: Newsletter 日报
icon: ✉️
feeds:
- name: Ben's Bites
url: https://bensbites.substack.com/feed
- name: The Rundown AI
url: https://therundown.substack.com/feed
- name: Latent Space
url: https://www.latent.space/feed
- name: Simon Willison
url: https://simonwillison.net/atom/everything/
- name: Import AI
url: https://importai.substack.com/feed
- name: Last Week in AI
url: https://lastweekin.ai/feed
- name: The Neuron
url: https://www.theneuron.ai/feed
- id: research
name: 研究 / 论文
icon: 📚
feeds:
- name: arXiv cs.CL
url: https://arxiv.org/rss/cs.CL
- name: arXiv cs.AI
url: https://arxiv.org/rss/cs.AI
- name: arXiv cs.LG
url: https://arxiv.org/rss/cs.LG
- id: trending
name: 热点 / 趋势
icon: 🔥
feeds:
- name: Google News · AI
url: https://news.google.com/rss/search?q=artificial+intelligence+OR+LLM+OR+Claude+OR+GPT&hl=en-US&gl=US&ceid=US:en
- name: Google News · Technology
url: https://news.google.com/rss/headlines/section/topic/TECHNOLOGY?hl=en-US&gl=US&ceid=US:en
- name: Techmeme
url: https://www.techmeme.com/feed.xml
- name: HN · Front Page
url: https://hnrss.org/frontpage
- name: HN · 100+ Points
url: https://hnrss.org/newest?points=100
- name: Dev.to · AI
url: https://dev.to/feed/tag/ai
- name: Lobsters
url: https://lobste.rs/rss
- id: community
name: 社区讨论
icon: 💬
feeds:
- name: HN · AI/LLM/Agent
url: https://hnrss.org/newest?q=AI+OR+LLM+OR+Claude+OR+agent+OR+GPT+OR+Gemini
- name: Reddit · LLM/Claude/ML
url: https://old.reddit.com/r/LocalLLaMA+ClaudeAI+MachineLearning+OpenAI/.rss?limit=25
slow: true
cn:
title_keywords:
- 人工智能
- 大模型
- 智能体
- 多模态
- AIGC
- LLM
- GPT
- Claude
- Gemini
- ChatGPT
- OpenAI
- Anthropic
- Copilot
- Agent
- 'AI '
- ' AI'
- AI·
- AI业务
- AI模型
- AI助手
- AI工具
- AI编程
- AI 编程
- AI版
- AI Agent
- 推理模型
- 深度学习
- 机器学习
- Function Calling
categories:
- id: media
name: AI 专业媒体
icon: 📰
feeds:
- name: 量子位
url: https://www.qbitai.com/feed
- name: InfoQ 中文
url: https://www.infoq.cn/feed/AI
- id: tech
name: 综合科技
icon: 📱
feeds:
- name: 36氪
url: https://36kr.com/feed
ai_filter: true
- name: 雷锋网
url: https://www.leiphone.com/feed
- name: Google News · AI
url: https://news.google.com/rss/search?q=人工智能+OR+大模型+OR+Agent+OR+LLM&hl=zh-CN&gl=CN&ceid=CN:zh-Hans
- id: dev
name: 开发者社区
icon: 💻
feeds:
- name: 掘金
url: https://juejin.cn/rss
ai_filter: true

View File

@@ -1,10 +0,0 @@
# 标题或摘要命中任一词则丢弃(不区分大小写)
words:
- 赌博
- 六合彩
- 网赌
- 色情
- 裸聊
- 代孕
- 办证
- 刷单兼职

View File

@@ -5,6 +5,7 @@ from __future__ import annotations
import sys import sys
from daily.generate import main as generate_main from daily.generate import main as generate_main
from daily.scheduler import main as schedule_main
from daily.webhook import main as push_main from daily.webhook import main as push_main
@@ -14,7 +15,14 @@ def main() -> int:
return generate_main() return generate_main()
if cmd in {"push", "send", "webhook"}: if cmd in {"push", "send", "webhook"}:
return push_main(sys.argv[2:]) return push_main(sys.argv[2:])
print(f"未知命令: {cmd}\n用法: python -m daily [generate|push] [report_path]", file=sys.stderr) if cmd in {"schedule", "scheduler", "daemon"}:
return schedule_main()
print(
f"未知命令: {cmd}\n"
"用法: python -m daily [generate|push|schedule] [report_path]\n"
" python -m daily schedule [--once] [--dry-run]",
file=sys.stderr,
)
return 1 return 1

View File

@@ -59,11 +59,41 @@ def _extract_markdown(text: str) -> str:
def analyze_trends(llm_input: dict[str, Any], *, date_str: str) -> dict[str, Any] | None: def analyze_trends(llm_input: dict[str, Any], *, date_str: str) -> dict[str, Any] | None:
from daily.config import theme_ban_days
from daily.narrative_axis import (
enforce_narrative_axis,
load_recent_axes,
load_recent_theme_summaries,
pick_narrative_axis,
)
skill = _load_skill() skill = _load_skill()
featured_note = ""
if llm_input.get("featured_pick"):
featured_note = (
"\n输入已含 **featured_pick**(编辑指定今日首推);"
"top_picks.skill 必须以 featured_pick 为准;"
"why/opening 不得向读者提及「编辑指定」。\n"
)
used_axes = set(load_recent_axes(date_str))
axis = pick_narrative_axis(used_axes)
llm_input["required_narrative_axis"] = axis
llm_input["narrative_axis"] = axis
theme_ban = load_recent_theme_summaries(date_str, theme_ban_days())
ban_note = ""
if theme_ban:
ban_note = (
"\n近几日已用过的主题/导语(请软避开同类开场,勿原样复用):\n- "
+ "\n- ".join(theme_ban)
+ "\n"
)
system = ( system = (
f"{skill}\n\n" f"{skill}\n\n"
f"{featured_note}"
f"{ban_note}"
"当前执行 **Step 1趋势分析**。\n" "当前执行 **Step 1趋势分析**。\n"
"只输出 trends JSONheadline, opening, themes, top_picks, signals不要 Markdown。" f"**required_narrative_axis** = `{axis}`;输出 JSON 必须含 `narrative_axis` 且等于该值。\n"
"只输出 trends JSONheadline, opening, themes, top_picks, signals, narrative_axis不要 Markdown。"
) )
user = json.dumps(llm_input, ensure_ascii=False, indent=2) user = json.dumps(llm_input, ensure_ascii=False, indent=2)
try: try:
@@ -77,8 +107,9 @@ def analyze_trends(llm_input: dict[str, Any], *, date_str: str) -> dict[str, Any
if not parsed.get("headline") and not parsed.get("opening"): if not parsed.get("headline") and not parsed.get("opening"):
logger.warning("Agent Step1 JSON 无效") logger.warning("Agent Step1 JSON 无效")
return None return None
parsed = enforce_narrative_axis(parsed, axis)
save_json(trends_json_path(date_str), parsed) save_json(trends_json_path(date_str), parsed)
logger.info("Agent Step1 完成:%s", parsed.get("headline", "?")) logger.info("Agent Step1 完成:%s [%s]", parsed.get("headline", "?"), axis)
return parsed return parsed
@@ -91,8 +122,17 @@ def write_wecom_report(
updated: str, updated: str,
) -> str | None: ) -> str | None:
skill = _load_skill() skill = _load_skill()
featured_note = ""
if llm_input.get("featured_pick"):
featured_note = (
"\n输入 data 已含 **featured_pick**"
"今日首推区块须使用 featured_pick.why_today"
"链接行用 Markdown [标题](URL),勿用反引号裸 URL"
"读者可见文案不得出现「编辑指定」等元信息。\n"
)
system = ( system = (
f"{skill}\n\n" f"{skill}\n\n"
f"{featured_note}"
"当前执行 **Step 2撰写企微早报**。\n" "当前执行 **Step 2撰写企微早报**。\n"
f"日期={date_str},时间={time_str},数据截至={updated}\n" f"日期={date_str},时间={time_str},数据截至={updated}\n"
"只输出企微 Markdown 正文,不要代码块,不要 JSON。" "只输出企微 Markdown 正文,不要代码块,不要 JSON。"
@@ -125,20 +165,10 @@ def run_agent_workflow(
trends = analyze_trends(llm_input, date_str=date_str) trends = analyze_trends(llm_input, date_str=date_str)
if not trends: if not trends:
return None return None
md = write_wecom_report( return write_wecom_report(
llm_input, llm_input,
trends, trends,
date_str=date_str, date_str=date_str,
time_str=time_str, time_str=time_str,
updated=updated, updated=updated,
) )
if md is None:
logger.warning("Agent Step2 首次失败,重试一次")
md = write_wecom_report(
llm_input,
trends,
date_str=date_str,
time_str=time_str,
updated=updated,
)
return md

163
daily/board_history.py Normal file
View File

@@ -0,0 +1,163 @@
"""企微展示历史:读写 data.wecom_shown_keys与 movement_baseline 严格分离。"""
from __future__ import annotations
import json
import logging
import re
from datetime import datetime, timedelta
from pathlib import Path
from typing import Any
from daily.config import OUTPUT_DIR, board_dedup_days
from daily.delta import RECENT_BOARD_KEYS, skill_id
logger = logging.getLogger(__name__)
BOARD_KEYS = RECENT_BOARD_KEYS
_GITHUB_REPO_RE = re.compile(r"github\.com/([\w.-]+/[\w.-]+)", re.I)
_SKILL_SH_RE = re.compile(r"skills\.sh/([\w.-]+/[\w.-]+(?:/[\w.-]+)?)", re.I)
# 与企微正文榜单标题对齐;顺序用于切分相邻 section
_WECOM_SECTION_PATTERNS: tuple[tuple[str, re.Pattern[str]], ...] = (
("skills_trending", re.compile(r"Skills\s+Trending", re.I)),
("skills_hot", re.compile(r"Skills\s+Hot", re.I)),
("github_trending", re.compile(r"GitHub\s+Trending", re.I)),
("github_emerging", re.compile(r"GitHub\s+新兴", re.I)),
("github_topic", re.compile(r"Topic\s+", re.I)),
)
def extract_shown_keys(board: str, items: list[dict[str, Any]]) -> list[str]:
"""从最终展示 items 抽取稳定 identity key。
Skills 榜同时写入 skill id 与 source便于周去重按仓屏蔽。
"""
keys: list[str] = []
seen: set[str] = set()
for item in items:
if board.startswith("skills_"):
candidates = [skill_id(item), str(item.get("source") or "").strip()]
else:
candidates = [str(item.get("repo") or "")]
for key in candidates:
if not key or key in seen:
continue
seen.add(key)
keys.append(key)
return keys
def _keys_from_board_items(board: str, data: dict[str, Any]) -> set[str]:
if board == "github_topic":
topic = data.get("github_topic") or {}
items = topic.get("repos") if isinstance(topic, dict) else []
else:
items = data.get(board) or []
if not isinstance(items, list):
return set()
return set(extract_shown_keys(board, items))
def parse_wecom_shown_keys(md: str) -> dict[str, set[str]]:
"""从企微 Markdown 按榜单 section 解析已展示 keys冷启动兼容"""
out: dict[str, set[str]] = {board: set() for board in BOARD_KEYS}
if not (md or "").strip():
return out
hits: list[tuple[int, str]] = []
for board, pattern in _WECOM_SECTION_PATTERNS:
for match in pattern.finditer(md):
hits.append((match.start(), board))
if not hits:
return out
hits.sort(key=lambda x: x[0])
for idx, (start, board) in enumerate(hits):
end = hits[idx + 1][0] if idx + 1 < len(hits) else len(md)
chunk = md[start:end]
if board.startswith("skills_"):
out[board].update(_SKILL_SH_RE.findall(chunk))
else:
out[board].update(_GITHUB_REPO_RE.findall(chunk))
return out
def _load_shown_keys_for_day(path: Path, date_str: str) -> dict[str, set[str]] | None:
"""读一日历史:优先 wecom_shown_keys缺省则回退 wecom.md再回退 data 榜字段。"""
empty = {board: set() for board in BOARD_KEYS}
try:
payload = json.loads(path.read_text(encoding="utf-8"))
except (OSError, ValueError) as exc:
logger.warning("读取 wecom_shown_keys %s 失败:%s", path, exc)
return None
data = payload.get("data") if isinstance(payload, dict) else None
if not isinstance(data, dict):
return empty
out: dict[str, set[str]] = {board: set() for board in BOARD_KEYS}
shown = data.get("wecom_shown_keys")
if isinstance(shown, dict):
for board in BOARD_KEYS:
keys = shown.get(board) or []
if isinstance(keys, list):
out[board].update(str(k) for k in keys if k)
if any(out.values()):
return out
wecom_path = OUTPUT_DIR / f"{date_str}.wecom.md"
if wecom_path.exists():
try:
md = wecom_path.read_text(encoding="utf-8")
except OSError as exc:
logger.warning("读取 wecom.md 回退 %s 失败:%s", wecom_path, exc)
else:
parsed = parse_wecom_shown_keys(md)
if any(parsed.values()):
return parsed
for board in BOARD_KEYS:
out[board].update(_keys_from_board_items(board, data))
return out
def load_recent_shown_keys(
date_str: str,
*,
lookback_days: int | None = None,
) -> dict[str, set[str]]:
"""近 N 日已展示 keys 并集(不含当日)。缺省或读失败视为空集。"""
empty = {board: set() for board in BOARD_KEYS}
try:
dt = datetime.strptime(date_str, "%Y-%m-%d")
except ValueError:
return empty
days = lookback_days if lookback_days is not None else board_dedup_days()
out: dict[str, set[str]] = {board: set() for board in BOARD_KEYS}
for day_offset in range(1, days + 1):
prev_date = (dt - timedelta(days=day_offset)).strftime("%Y-%m-%d")
path = OUTPUT_DIR / f"{prev_date}.data.json"
if not path.exists():
continue
day_keys = _load_shown_keys_for_day(path, prev_date)
if day_keys is None:
continue
for board in BOARD_KEYS:
out[board].update(day_keys.get(board) or set())
return out
def merge_wecom_shown_into_data(
data: dict[str, Any],
shown: dict[str, list[str]],
) -> dict[str, Any]:
"""写入 wecom_shown_keys不修改 movement_baseline。"""
merged = dict(data)
merged["wecom_shown_keys"] = {
board: list(keys) for board, keys in shown.items()
}
return merged

48
daily/board_select.py Normal file
View File

@@ -0,0 +1,48 @@
"""五榜唯一列表主人:周去重 + 深池补满。"""
from __future__ import annotations
import logging
from typing import Any, Literal
from daily.delta import skill_id
logger = logging.getLogger(__name__)
def board_select(
*,
board: str,
items: list[dict[str, Any]],
recent_keys: set[str],
limit: int,
pool_size: int,
kind: Literal["skill", "github"],
) -> list[dict[str, Any]]:
"""从深池过滤近 N 日已展示 key按原顺序取满 limit不足则短榜。"""
if kind == "skill":
from daily.skills_group import group_skills_by_source
pool = group_skills_by_source(items, limit=pool_size, pool_size=pool_size)
else:
pool = items[: max(pool_size, limit)]
out: list[dict[str, Any]] = []
for item in pool:
if kind == "skill":
key = skill_id(item)
source = str(item.get("source") or "").strip()
if (key and key in recent_keys) or (source and source in recent_keys):
continue
if not key and not source:
continue
else:
key = str(item.get("repo") or "")
if not key or key in recent_keys:
continue
out.append(item)
if len(out) >= limit:
break
if len(out) < limit:
logger.info("board_short:%s:%s", board, len(out))
return out

View File

@@ -12,7 +12,7 @@ import time
from pathlib import Path from pathlib import Path
from typing import Any, Mapping from typing import Any, Mapping
import env_config from daily.config import ROOT, env
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -22,7 +22,7 @@ _bridge_process: subprocess.Popen[bytes] | None = None
def _cursor_cwd() -> str: def _cursor_cwd() -> str:
return env_config.env("CURSOR_CWD", r"d:\LY\test\tech") or r"d:\LY\test\tech" return env("DAILY_CURSOR_CWD") or env("CURSOR_CWD") or str(ROOT)
def _parse_discovery_line(line: str) -> Mapping[str, Any] | None: def _parse_discovery_line(line: str) -> Mapping[str, Any] | None:
@@ -124,7 +124,8 @@ def warm_cursor_bridge(force: bool = False) -> None:
) )
try: try:
discovery = _read_discovery_polling(process) discovery = _read_discovery_polling(process)
except Exception: except (RuntimeError, OSError, ValueError) as exc:
logger.warning("Cursor bridge discovery 失败,终止子进程:%s", exc)
process.kill() process.kill()
process.wait(timeout=5) process.wait(timeout=5)
raise raise

View File

@@ -30,6 +30,24 @@ def wecom_skill_desc_limit() -> int:
return env_int("DAILY_WECOM_SKILL_DESC_LIMIT", 56) return env_int("DAILY_WECOM_SKILL_DESC_LIMIT", 56)
def wecom_news_desc_limit() -> int:
"""企微新闻摘要建议字数;在句读/词边界截断,不加省略号。"""
return max(24, env_int("DAILY_WECOM_NEWS_DESC_LIMIT", 72))
def wecom_ai_news_tech_limit() -> int:
"""research 模式下技术类时讯条数(叠加在 DAILY_WECOM_AI_NEWS 之上)。"""
return max(0, env_int("DAILY_WECOM_AI_NEWS_TECH", 5))
def wecom_pad_pool_size(display_limit: int) -> int:
"""Delta 补榜候选池大小(展示条数之上多取,避免去重后凑不满)。"""
explicit = env_int("DAILY_WECOM_PAD_POOL", -1)
if explicit > 0:
return explicit
return max(display_limit * 5, 50)
def full_desc_limit() -> int: def full_desc_limit() -> int:
"""完整版早报摘要长度0 表示不截断。""" """完整版早报摘要长度0 表示不截断。"""
return env_int("DAILY_FULL_DESC_LIMIT", 0) return env_int("DAILY_FULL_DESC_LIMIT", 0)
@@ -43,6 +61,7 @@ def wecom_max_bytes() -> int:
"""兼容旧配置名。""" """兼容旧配置名。"""
return wecom_chunk_bytes() return wecom_chunk_bytes()
load_dotenv(ROOT / ".env") load_dotenv(ROOT / ".env")
load_dotenv(ROOT / ".env.local", override=True) load_dotenv(ROOT / ".env.local", override=True)
@@ -71,3 +90,80 @@ def env_int(key: str, default: int) -> int:
return int(raw) return int(raw)
except ValueError: except ValueError:
return default return default
def env_bool(key: str, default: bool) -> bool:
raw = env(key)
if raw is None:
return default
return raw.strip().lower() in {"1", "true", "yes", "on"}
def wecom_mode() -> str:
raw = (env("DAILY_WECOM_MODE") or "delta").strip().lower()
return raw if raw in {"full", "delta"} else "delta"
def news_dedup_days() -> int:
return max(1, env_int("DAILY_NEWS_DEDUP_DAYS", 7))
def skip_push_when_silent() -> bool:
return env_bool("DAILY_SKIP_PUSH_WHEN_SILENT", True)
def delta_baseline_fallback() -> str:
raw = (env("DAILY_DELTA_BASELINE_FALLBACK") or "full").strip().lower()
return raw if raw in {"full", "empty"} else "full"
def wecom_delta_pad() -> bool:
"""Delta 模式下新入榜优先,不足时用当日 Top 榜补满;补榜排除近 N 天 baseline 已出现条目。"""
return env_bool("DAILY_WECOM_DELTA_PAD", True)
def delta_pad_lookback_days() -> int:
"""补榜时排除近 N 天 baseline 已出现过的条目(默认与异动对比窗口一致)。"""
fallback = env_int("DAILY_DELTA_LOOKBACK_DAYS", 7)
return max(1, env_int("DAILY_DELTA_PAD_LOOKBACK_DAYS", fallback))
def force_push() -> bool:
return env_bool("DAILY_FORCE_PUSH", False)
def schedule_timezone_name() -> str:
return (env("DAILY_SCHEDULE_TZ") or "Asia/Shanghai").strip() or "Asia/Shanghai"
def schedule_generate_at() -> str:
return (env("DAILY_SCHEDULE_GENERATE_AT") or "08:50").strip() or "08:50"
def schedule_push_at() -> str:
return (env("DAILY_SCHEDULE_PUSH_AT") or "09:00").strip() or "09:00"
def board_dedup_days() -> int:
return max(1, env_int("DAILY_BOARD_DEDUP_DAYS", 7))
def board_pool_size() -> int:
fallback = env_int("DAILY_WECOM_SKILL_POOL", 400)
return max(1, env_int("DAILY_BOARD_POOL_SIZE", max(200, fallback)))
def featured_dedup_days() -> int:
return max(1, env_int("DAILY_FEATURED_DEDUP_DAYS", 30))
def theme_ban_days() -> int:
return max(1, env_int("DAILY_THEME_BAN_DAYS", 7))
def narrative_axis_days() -> int:
return max(1, env_int("DAILY_NARRATIVE_AXIS_DAYS", 3))
def news_backfill_enabled() -> bool:
return env_bool("DAILY_NEWS_BACKFILL", False)

View File

@@ -1,142 +0,0 @@
"""Windows 兼容的 Cursor SDK bridgedaily 包自用,不依赖 bot/)。"""
from __future__ import annotations
import codecs
import json
import logging
import os
import subprocess
import threading
import time
from pathlib import Path
from typing import Any, Mapping
from daily.config import ROOT, env
logger = logging.getLogger(__name__)
READY_LINE_PREFIX = "cursor-sdk-bridge ready "
_bridge_lock = threading.Lock()
_bridge_process: subprocess.Popen[bytes] | None = None
def cursor_cwd() -> str:
return env("DAILY_CURSOR_CWD") or env("CURSOR_CWD") or str(ROOT)
def _parse_discovery_line(line: str) -> Mapping[str, Any] | None:
if not line.startswith(READY_LINE_PREFIX):
return None
payload = line[len(READY_LINE_PREFIX) :].strip()
loaded = json.loads(payload)
if not isinstance(loaded, dict):
raise RuntimeError("Bridge discovery payload must be an object")
return loaded
def _read_discovery_polling(process: subprocess.Popen[bytes], timeout: float = 60) -> Mapping[str, Any]:
if process.stderr is None:
raise RuntimeError("Bridge stderr unavailable")
fd = process.stderr.fileno()
was_blocking = os.get_blocking(fd)
os.set_blocking(fd, False)
decoder = codecs.getincrementaldecoder("utf-8")(errors="replace")
pending = ""
stderr_lines: list[str] = []
deadline = time.monotonic() + timeout
try:
while time.monotonic() < deadline:
try:
chunk = os.read(fd, 8192)
except BlockingIOError:
chunk = b""
if chunk:
pending += decoder.decode(chunk)
while "\n" in pending:
line, pending = pending.split("\n", 1)
stderr_lines.append(line)
discovery = _parse_discovery_line(line)
if discovery is not None:
return discovery
else:
code = process.poll()
if code is not None:
pending += decoder.decode(b"", final=True)
if pending.strip():
stderr_lines.append(pending.strip())
joined = "\n".join(stderr_lines)[-2000:]
raise RuntimeError(
f"Bridge 启动失败 exit={code}: {joined or '无 stderr 输出'}"
)
time.sleep(0.05)
finally:
os.set_blocking(fd, was_blocking)
raise RuntimeError("等待 Cursor bridge 就绪超时")
def _auth_token_from_discovery(discovery: Mapping[str, Any]) -> str:
token = str(discovery.get("authToken") or "").strip()
if token:
return token
token_file = discovery.get("authTokenFile")
if token_file:
return Path(str(token_file)).read_text(encoding="utf-8").strip()
raise RuntimeError("Bridge discovery 缺少 auth token")
def warm_cursor_bridge(force: bool = False) -> None:
"""启动 cursor-sdk-bridge 并写入 CURSOR_SDK_BRIDGE_* 环境变量。"""
global _bridge_process
with _bridge_lock:
if (
not force
and _bridge_process is not None
and _bridge_process.poll() is None
and os.environ.get("CURSOR_SDK_BRIDGE_URL")
and os.environ.get("CURSOR_SDK_BRIDGE_TOKEN")
):
return
if _bridge_process is not None and _bridge_process.poll() is None:
_bridge_process.terminate()
try:
_bridge_process.wait(timeout=5)
except subprocess.TimeoutExpired:
_bridge_process.kill()
from cursor_sdk._vendor import resolve_bridge_path
cwd = cursor_cwd()
os.environ["CURSOR_CWD"] = cwd
argv = [resolve_bridge_path(), "--workspace", cwd]
logger.info("启动 Cursor bridge workspace=%s", cwd)
process = subprocess.Popen(
argv,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
)
try:
discovery = _read_discovery_polling(process)
except Exception:
process.kill()
process.wait(timeout=5)
raise
url = str(discovery.get("url") or "").strip()
if not url:
host = str(discovery.get("host") or "127.0.0.1")
port = discovery.get("port")
url = f"http://{host}:{port}"
token = _auth_token_from_discovery(discovery)
os.environ["CURSOR_SDK_BRIDGE_URL"] = url
os.environ["CURSOR_SDK_BRIDGE_TOKEN"] = token
_bridge_process = process
logger.info("Cursor bridge 就绪: %s", url)

View File

@@ -1,68 +0,0 @@
"""Cursor SDK 调用daily 包专用)。"""
from __future__ import annotations
import os
from cursor_sdk import Agent, AgentOptions, Client, CursorAgentError, LocalAgentOptions
from daily.config import env, env_int
from daily.cursor_bridge import cursor_cwd, warm_cursor_bridge
_sdk_client: Client | None = None
_sdk_client_key: tuple[str, str, float] | None = None
def cursor_timeout_seconds() -> float:
"""Cursor bridge unary/stream 超时(秒);与 bot 共用 CURSOR_TIMEOUT。"""
return float(env_int("CURSOR_TIMEOUT", 600))
def _cursor_sdk_client() -> Client:
"""带自定义超时的 bridge ClientSDK 默认 unary 仅 60s复杂 Agent 任务易超时)。"""
global _sdk_client, _sdk_client_key
warm_cursor_bridge()
url = os.environ.get("CURSOR_SDK_BRIDGE_URL", "")
token = os.environ.get("CURSOR_SDK_BRIDGE_TOKEN", "")
timeout = cursor_timeout_seconds()
key = (url, token, timeout)
if _sdk_client is not None and _sdk_client_key == key:
return _sdk_client
if _sdk_client is not None:
_sdk_client.close()
_sdk_client = Client(
base_url=url,
auth_token=token,
unary_timeout=timeout,
stream_timeout=timeout,
allow_api_key_env_fallback=False,
)
_sdk_client_key = key
return _sdk_client
def cursor_chat(system: str, user: str) -> str:
api_key = (env("CURSOR_API_KEY") or "").strip()
if not api_key:
return ""
client = _cursor_sdk_client()
cwd = cursor_cwd()
model = env("CURSOR_MODEL") or "composer-2.5"
prompt = f"{system}\n\n{user}"
try:
result = Agent.prompt(
prompt,
AgentOptions(
api_key=api_key,
model=model,
local=LocalAgentOptions(cwd=cwd),
),
client=client,
)
except CursorAgentError as exc:
raise RuntimeError(f"LLM 调用失败:{exc.message}") from exc
if result.status == "error":
raise RuntimeError(f"LLM 调用失败:{result.result or '未知错误'}")
return (result.result or "").strip()

View File

@@ -8,13 +8,22 @@ from datetime import datetime, timedelta
from pathlib import Path from pathlib import Path
from typing import Any, Callable from typing import Any, Callable
from daily.config import OUTPUT_DIR, env_int from daily.config import OUTPUT_DIR, delta_baseline_fallback, env_int, wecom_mode
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
KeyFn = Callable[[dict[str, Any]], str] KeyFn = Callable[[dict[str, Any]], str]
RECENT_BOARD_KEYS = (
"skills_trending",
"skills_hot",
"github_trending",
"github_emerging",
"github_topic",
)
def compare_depth() -> int: def compare_depth() -> int:
return env_int("DAILY_DELTA_COMPARE_DEPTH", 15) return env_int("DAILY_DELTA_COMPARE_DEPTH", 15)
@@ -39,6 +48,23 @@ def _key_set(items: list[dict[str, Any]], key_fn: KeyFn, *, depth: int) -> set[s
return {key_fn(item) for item in items[:depth] if key_fn(item)} return {key_fn(item) for item in items[:depth] if key_fn(item)}
def load_recent_board_keys(
date_str: str,
*,
lookback_days: int | None = None,
) -> dict[str, set[str]]:
"""近 N 天各榜已展示过的 skill id / repo不含当日供补榜去重
委托 board_history.load_recent_shown_keys只读 wecom_shown_keys
不读 movement_baseline。
"""
from daily.board_history import load_recent_shown_keys
from daily.config import board_dedup_days
days = lookback_days if lookback_days is not None else board_dedup_days()
return load_recent_shown_keys(date_str, lookback_days=days)
def find_previous_data(date_str: str) -> tuple[str, dict[str, Any]] | None: def find_previous_data(date_str: str) -> tuple[str, dict[str, Any]] | None:
"""查找最近一份早于 date_str 的 data.json。""" """查找最近一份早于 date_str 的 data.json。"""
try: try:
@@ -311,3 +337,40 @@ def build_movement_context(
"skills_stable": not skills_trending_all and not skills_hot_all, "skills_stable": not skills_trending_all and not skills_hot_all,
"github_stable": not github_trending_all and not github_emerging_all and not github_topic_all, "github_stable": not github_trending_all and not github_emerging_all and not github_topic_all,
} }
def effective_wecom_mode(*, date_str: str, configured_mode: str | None = None) -> str:
mode = configured_mode or wecom_mode()
if mode != "delta":
return "full"
if find_previous_data(date_str) is None and delta_baseline_fallback() == "full":
return "full"
return "delta"
def partition_skill_moves_for_wecom(
trending_moves: list[dict[str, Any]],
hot_moves: list[dict[str, Any]],
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
hot_by_id = {skill_id(m): m for m in hot_moves if skill_id(m)}
trending_out: list[dict[str, Any]] = []
consumed_hot: set[str] = set()
for move in trending_moves:
sid = skill_id(move)
copy = dict(move)
badges = [f"Trending #{move.get('rank', '?')}"]
hot_match = hot_by_id.get(sid)
if hot_match:
badges.append(f"Hot #{hot_match.get('rank', '?')}")
consumed_hot.add(sid)
copy["badge"] = " · ".join(badges)
trending_out.append(copy)
hot_out: list[dict[str, Any]] = []
for move in hot_moves:
sid = skill_id(move)
if sid in consumed_hot:
continue
copy = dict(move)
copy["badge"] = f"Hot #{move.get('rank', '?')}"
hot_out.append(copy)
return trending_out, hot_out

545
daily/featured_pick.py Normal file
View File

@@ -0,0 +1,545 @@
"""今日首推:解析 DAILY_FEATURED_PICK → 定人(月去重)→ LLM 检索 → featured JSON。"""
from __future__ import annotations
import hashlib
import json
import logging
import random
import re
from datetime import datetime, timedelta
from typing import Any
from daily.config import OUTPUT_DIR, ROOT, env, featured_dedup_days
from daily.llm_client import extract_json_object, has_llm_configured, llm_chat
from daily.report_data import featured_json_path, save_json
logger = logging.getLogger(__name__)
_SKILL_DIR = ROOT / "skills" / "daily-featured-pick"
_GITHUB_REPO_RE = re.compile(r"github\.com/([^/\s#?]+/[^/\s#?]+)", re.I)
def featured_identity_key(featured: dict[str, Any] | None) -> str:
"""稳定身份skill→idgithub→repo兜底从 url 解析。"""
if not featured:
return ""
typ = str(featured.get("type") or "").lower()
if typ == "skill" or featured.get("id"):
sid = str(featured.get("id") or "").strip()
if sid:
return sid
repo = str(featured.get("repo") or "").strip()
if repo:
return repo
for field in ("url", "command", "link"):
url = str(featured.get(field) or "")
m = _GITHUB_REPO_RE.search(url)
if m:
return m.group(1)
return ""
def load_recent_featured_keys(date_str: str, days: int | None = None) -> set[str]:
"""近 N 日 data.featured_pick_key 并集(不含当日)。"""
lookback = days if days is not None else featured_dedup_days()
try:
dt = datetime.strptime(date_str, "%Y-%m-%d")
except ValueError:
return set()
out: set[str] = set()
for day_offset in range(1, lookback + 1):
prev = (dt - timedelta(days=day_offset)).strftime("%Y-%m-%d")
path = OUTPUT_DIR / f"{prev}.data.json"
if not path.exists():
continue
try:
payload = json.loads(path.read_text(encoding="utf-8"))
except (OSError, ValueError) as exc:
logger.warning("读取 featured_pick_key %s 失败:%s", path, exc)
continue
data = payload.get("data") if isinstance(payload, dict) else None
if not isinstance(data, dict):
continue
key = str(data.get("featured_pick_key") or "").strip()
if not key:
featured = data.get("featured_pick")
if isinstance(featured, dict):
key = featured_identity_key(featured)
if key:
out.add(key)
return out
def load_yesterday_featured_key(date_str: str) -> str | None:
try:
dt = datetime.strptime(date_str, "%Y-%m-%d")
except ValueError:
return None
prev = (dt - timedelta(days=1)).strftime("%Y-%m-%d")
path = OUTPUT_DIR / f"{prev}.data.json"
if not path.exists():
return None
try:
payload = json.loads(path.read_text(encoding="utf-8"))
except (OSError, ValueError):
return None
data = payload.get("data") if isinstance(payload, dict) else None
if not isinstance(data, dict):
return None
key = str(data.get("featured_pick_key") or "").strip()
if key:
return key
featured = data.get("featured_pick")
if isinstance(featured, dict):
return featured_identity_key(featured) or None
return None
def _featured_rng(date_str: str) -> random.Random:
seed = int(hashlib.sha256(f"{date_str}:featured".encode()).hexdigest()[:16], 16)
return random.Random(seed)
def _stub_from_pool_item(item: dict[str, Any]) -> dict[str, Any]:
repo = str(item.get("repo") or "").strip()
if repo:
url = str(item.get("url") or f"https://github.com/{repo}").strip()
return {
"type": "github",
"title": repo.split("/")[-1],
"repo": repo,
"url": url,
"command": url,
"summary": str(item.get("description") or "")[:160],
"why_today": "",
"evidence": [],
"tags": [],
}
sid = str(item.get("id") or "").strip()
source = str(item.get("source") or "").strip()
title = str(item.get("title") or "").strip()
return {
"type": "skill",
"id": sid,
"title": title or sid,
"command": _skill_command(item),
"url": str(item.get("link") or ""),
"summary": str(item.get("description") or "")[:160],
"why_today": "",
"evidence": [],
"tags": [],
"source": source,
}
def featured_resolve(
*,
date_str: str,
candidate: dict[str, Any] | None,
pool_a: list[dict[str, Any]],
pool_b: list[dict[str, Any]],
recent_featured: set[str] | None = None,
yesterday_key: str | None = None,
rng: random.Random | None = None,
) -> tuple[dict[str, Any] | None, str | None]:
"""若与昨日同一身份则改推;返回 (seed_stub, identity_key),不含完整 why。"""
if not candidate:
return None, None
key = featured_identity_key(candidate)
if not yesterday_key or key != yesterday_key:
return candidate, key or None
blocked = set(recent_featured or set()) | {yesterday_key}
picker = rng or _featured_rng(date_str)
def _choices(pool: list[dict[str, Any]]) -> list[tuple[str, dict[str, Any]]]:
out: list[tuple[str, dict[str, Any]]] = []
seen: set[str] = set()
for item in pool:
ik = featured_identity_key(item)
if not ik or ik in blocked or ik in seen:
continue
seen.add(ik)
out.append((ik, item))
return out
options = _choices(pool_a)
if not options:
options = _choices(pool_b)
if not options:
logger.info("featured_fallback_exhausted")
return candidate, key
chosen_key, chosen_item = picker.choice(options)
return _stub_from_pool_item(chosen_item), chosen_key
def _config_from_candidate(candidate: dict[str, Any]) -> dict[str, str]:
typ = str(candidate.get("type") or "").lower()
if typ == "skill" or candidate.get("id"):
query = str(candidate.get("id") or candidate.get("title") or "").strip()
return {"query": query, "url_hint": str(candidate.get("url") or "")}
repo = str(candidate.get("repo") or "").strip()
if repo:
return {
"query": repo,
"url_hint": str(candidate.get("url") or f"https://github.com/{repo}"),
}
query = str(candidate.get("title") or candidate.get("url") or "").strip()
return {"query": query or "featured", "url_hint": str(candidate.get("url") or "")}
def _seed_candidate_from_config(
config: dict[str, str],
llm_input: dict[str, Any],
) -> dict[str, Any]:
matches = match_in_data(llm_input, config["query"])
if matches["skills"]:
return _stub_from_pool_item(matches["skills"][0])
if matches["github"]:
return _stub_from_pool_item(matches["github"][0])
url = config.get("url_hint") or ""
seed: dict[str, Any] = {
"type": "other",
"title": config["query"],
"url": url,
"command": url or config["query"],
}
m = _GITHUB_REPO_RE.search(url)
if m:
seed["type"] = "github"
seed["repo"] = m.group(1)
return seed
def parse_featured_pick() -> dict[str, str] | None:
"""解析 DAILY_FEATURED_PICKquery 或 query|url。"""
raw = (env("DAILY_FEATURED_PICK") or "").strip()
if not raw:
return None
if "|" in raw:
query, url_hint = raw.split("|", 1)
query = query.strip()
url_hint = url_hint.strip()
if not query:
return None
payload: dict[str, str] = {"query": query}
if url_hint:
payload["url_hint"] = url_hint
return payload
return {"query": raw}
def _matches_query(text: str, query: str) -> bool:
return query.lower() in (text or "").lower()
def _skill_matches(item: dict[str, Any], query: str) -> bool:
for key in ("id", "title", "source"):
if _matches_query(str(item.get(key) or ""), query):
return True
for sub in item.get("cluster_skills") or []:
if isinstance(sub, str) and _matches_query(sub, query):
return True
return False
def match_in_data(llm_input: dict[str, Any], query: str) -> dict[str, list[dict[str, Any]]]:
"""在榜单数据中模糊匹配 query。"""
skills: list[dict[str, Any]] = []
seen_skill: set[str] = set()
for board in ("skills_trending", "skills_hot"):
for item in llm_input.get(board) or []:
sid = str(item.get("id") or "")
if sid in seen_skill:
continue
if _skill_matches(item, query):
seen_skill.add(sid)
skills.append({**item, "board": board})
if len(skills) >= 5:
break
if len(skills) >= 5:
break
github: list[dict[str, Any]] = []
seen_repo: set[str] = set()
for board in ("github_trending", "github_emerging"):
for item in llm_input.get(board) or []:
repo = str(item.get("repo") or "")
if not repo or repo in seen_repo:
continue
if _matches_query(repo, query):
seen_repo.add(repo)
github.append({**item, "board": board})
if len(github) >= 5:
break
topic = llm_input.get("github_topic") or {}
for item in topic.get("repos") or []:
repo = str(item.get("repo") or "")
if not repo or repo in seen_repo:
continue
if _matches_query(repo, query):
seen_repo.add(repo)
github.append({**item, "board": "github_topic"})
if len(github) >= 5:
break
return {"skills": skills, "github": github}
def _load_skill() -> str:
path = _SKILL_DIR / "SKILL.md"
if path.exists():
return path.read_text(encoding="utf-8").strip()
return "你是早报编辑。根据输入检索今日首推信息,只输出 JSON。"
def _skill_command(item: dict[str, Any]) -> str:
source = str(item.get("source") or "").strip()
title = str(item.get("title") or "").strip()
if source and title:
return f"npx skills add {source}/{title}"
sid = str(item.get("id") or "").strip()
if sid.count("/") >= 2:
parts = sid.split("/", 2)
return f"npx skills add {parts[0]}/{parts[1]}/{parts[2]}"
if sid.count("/") == 1:
return f"npx skills add {sid}"
return ""
def _evidence_from_skill(item: dict[str, Any]) -> list[str]:
board = item.get("board", "")
board_label = {
"skills_trending": "Skills Trending",
"skills_hot": "Skills Hot",
}.get(str(board), str(board))
installs = item.get("installs_fmt") or item.get("installs")
title = item.get("title") or item.get("id") or "?"
if installs:
return [f"{board_label} 匹配 · {title} · {installs}"]
return [f"{board_label} 匹配 · {title}"]
def _evidence_from_github(item: dict[str, Any]) -> list[str]:
board = item.get("board", "")
board_label = {
"github_trending": "GitHub Trending",
"github_emerging": "GitHub 新兴",
"github_topic": "GitHub Topic",
}.get(str(board), str(board))
repo = item.get("repo") or "?"
stars = item.get("total_stars_fmt") or ""
if stars:
return [f"{board_label} 匹配 · {repo} · ⭐{stars}"]
return [f"{board_label} 匹配 · {repo}"]
def _fallback_featured(
config: dict[str, str],
llm_input: dict[str, Any],
*,
partial: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""LLM 不可用或失败时,用榜单匹配 + 配置回退。"""
partial = partial or {}
matches = match_in_data(llm_input, config["query"])
skill = matches["skills"][0] if matches["skills"] else None
gh = matches["github"][0] if matches["github"] else None
if skill:
featured: dict[str, Any] = {
"title": str(skill.get("title") or config["query"]),
"type": "skill",
"command": _skill_command(skill),
"url": str(skill.get("link") or config.get("url_hint") or ""),
"summary": str(partial.get("summary") or skill.get("description") or "")[:160],
"why_today": str(
partial.get("why_today")
or f"今日 Skills 榜匹配到 **{skill.get('title') or config['query']}**,适合作为首推。"
),
"evidence": list(partial.get("evidence") or _evidence_from_skill(skill)),
"tags": list(partial.get("tags") or []),
}
if skill.get("id"):
featured["id"] = skill["id"]
return featured
if gh:
return {
"title": str(gh.get("repo") or config["query"]).split("/")[-1],
"type": "github",
"command": str(gh.get("url") or config.get("url_hint") or ""),
"url": str(gh.get("url") or config.get("url_hint") or ""),
"summary": str(partial.get("summary") or gh.get("description") or "")[:160],
"why_today": str(
partial.get("why_today")
or f"今日 GitHub 榜匹配到 **{gh.get('repo')}**,适合作为首推。"
),
"evidence": list(partial.get("evidence") or _evidence_from_github(gh)),
"tags": list(partial.get("tags") or []),
"repo": gh.get("repo"),
}
url = config.get("url_hint") or ""
return {
"title": config["query"],
"type": "other",
"command": url or config["query"],
"url": url,
"summary": str(partial.get("summary") or "")[:160],
"why_today": str(
partial.get("why_today")
or f"**{config['query']}** 未出现在今日 Top 榜,仍值得单独关注。"
),
"evidence": list(partial.get("evidence") or ([f"主推 · {config['query']}"])),
"tags": list(partial.get("tags") or []),
}
def _normalize_featured(
raw: dict[str, Any],
config: dict[str, str],
llm_input: dict[str, Any],
) -> dict[str, Any]:
"""补齐 command / url / evidence并与榜单数字对齐。"""
matches = match_in_data(llm_input, config["query"])
skill = matches["skills"][0] if matches["skills"] else None
gh = matches["github"][0] if matches["github"] else None
featured = dict(raw)
featured.setdefault("title", config["query"])
featured.setdefault("type", "other")
if skill and featured.get("type") in {"skill", "other", ""}:
featured.setdefault("id", skill.get("id"))
featured.setdefault("command", _skill_command(skill))
featured.setdefault("url", skill.get("link") or config.get("url_hint") or "")
if not featured.get("evidence"):
featured["evidence"] = _evidence_from_skill(skill)
featured["type"] = "skill"
elif gh and featured.get("type") in {"github", "other", ""}:
featured.setdefault("repo", gh.get("repo"))
featured.setdefault("url", gh.get("url") or config.get("url_hint") or "")
featured.setdefault("command", featured.get("url") or gh.get("url") or "")
if not featured.get("evidence"):
featured["evidence"] = _evidence_from_github(gh)
featured["type"] = "github"
featured.setdefault("command", config.get("url_hint") or config["query"])
featured.setdefault("url", config.get("url_hint") or "")
featured.setdefault("summary", "")
featured.setdefault("why_today", featured.get("summary") or "")
featured.setdefault("evidence", [])
featured.setdefault("tags", [])
return featured
def research_featured_pick(
llm_input: dict[str, Any],
*,
date_str: str,
config: dict[str, str] | None = None,
) -> dict[str, Any] | None:
"""Step 0检索今日首推成功返回 featured dict未配置返回 None。"""
config = config or parse_featured_pick()
if not config:
return None
matches = match_in_data(llm_input, config["query"])
payload = {
"query": config["query"],
"url_hint": config.get("url_hint"),
"cwd": env("DAILY_CURSOR_CWD") or str(ROOT),
"data_matches": matches,
}
if not has_llm_configured():
featured = _fallback_featured(config, llm_input)
save_json(featured_json_path(date_str), featured)
logger.info("Featured pick无 LLM规则回退%s", featured.get("title"))
return featured
skill = _load_skill()
system = (
f"{skill}\n\n"
"当前执行 **Step 0今日首推检索**。\n"
"只输出 featured JSONtitle, type, command, url, summary, why_today, evidence, tags"
"不要 Markdown不要解释。"
)
user = json.dumps(payload, ensure_ascii=False, indent=2)
try:
raw = llm_chat(system, user)
except Exception as exc:
logger.warning("Featured pick LLM 失败,回退规则模式:%s", exc)
featured = _fallback_featured(config, llm_input)
save_json(featured_json_path(date_str), featured)
return featured
if not raw:
featured = _fallback_featured(config, llm_input)
save_json(featured_json_path(date_str), featured)
return featured
parsed = extract_json_object(raw)
if not parsed.get("why_today") and not parsed.get("summary"):
logger.warning("Featured pick JSON 无效,回退规则模式")
featured = _fallback_featured(config, llm_input, partial=parsed)
save_json(featured_json_path(date_str), featured)
return featured
featured = _normalize_featured(parsed, config, llm_input)
save_json(featured_json_path(date_str), featured)
logger.info("Featured pick 完成:%s", featured.get("title"))
return featured
def apply_featured_pick(
llm_input: dict[str, Any],
*,
date_str: str,
pool_a: list[dict[str, Any]] | None = None,
pool_b: list[dict[str, Any]] | None = None,
) -> dict[str, Any] | None:
"""先定人(相对昨日改推 + 月去重),再 research写入 featured_pick / featured_pick_key。"""
config = parse_featured_pick()
if not config:
return None
seed = _seed_candidate_from_config(config, llm_input)
recent = load_recent_featured_keys(date_str)
yesterday = load_yesterday_featured_key(date_str)
resolved, identity_key = featured_resolve(
date_str=date_str,
candidate=seed,
pool_a=pool_a or [],
pool_b=pool_b or [],
recent_featured=recent,
yesterday_key=yesterday,
rng=_featured_rng(date_str),
)
research_config = config
if resolved and identity_key and featured_identity_key(seed) != identity_key:
research_config = _config_from_candidate(resolved)
featured = research_featured_pick(
llm_input, date_str=date_str, config=research_config
)
if featured:
llm_input["featured_pick"] = featured
key = identity_key or featured_identity_key(featured)
if key:
llm_input["featured_pick_key"] = key
return featured
def pick_command_from_featured(featured: dict[str, Any] | None) -> str | None:
cmd = str((featured or {}).get("command") or "").strip()
return cmd or None
def pick_why_from_featured(featured: dict[str, Any] | None) -> str | None:
why = str((featured or {}).get("why_today") or "").strip()
return why or None

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -43,38 +43,59 @@ def search_github_repos(
logger.warning("GitHub Search 需要 GITHUB_TOKEN: %s", query[:80]) logger.warning("GitHub Search 需要 GITHUB_TOKEN: %s", query[:80])
return [] return []
target = max(1, min(int(limit), 1000))
per_page = min(100, target)
repos: list[dict[str, Any]] = []
seen: set[str] = set()
page = 1
try: try:
with httpx.Client( with httpx.Client(
timeout=20.0, timeout=20.0,
verify=certifi.where(), verify=certifi.where(),
headers=github_api_headers(), headers=github_api_headers(),
) as client: ) as client:
resp = client.get( while len(repos) < target:
"https://api.github.com/search/repositories", resp = client.get(
params={ "https://api.github.com/search/repositories",
"q": query, params={
"sort": sort, "q": query,
"order": "desc", "sort": sort,
"per_page": min(max(limit, 1), 30), "order": "desc",
}, "per_page": per_page,
) "page": page,
if resp.status_code != 200: },
logger.warning("GitHub Search 失败 (%s): %s", resp.status_code, query[:80]) )
return [] if resp.status_code != 200:
items = resp.json().get("items") or [] logger.warning(
"GitHub Search 失败 (%s page=%s): %s",
resp.status_code,
page,
query[:80],
)
break
items = resp.json().get("items") or []
if not items:
break
for item in items:
full_name = item.get("full_name") or ""
if not full_name or full_name in seen:
continue
seen.add(full_name)
repos.append(_repo_from_api_item(item, source="api-search"))
if len(repos) >= target:
break
if len(items) < per_page:
break
page += 1
# Search API 最多约 1000 条 / 10 页
if page > 10:
break
except Exception as exc: except Exception as exc:
logger.warning("GitHub Search 异常: %s", exc) logger.warning("GitHub Search 异常: %s", exc)
return [] return repos[:target]
repos: list[dict[str, Any]] = [] return repos[:target]
for item in items:
full_name = item.get("full_name") or ""
if not full_name:
continue
repos.append(_repo_from_api_item(item, source="api-search"))
if len(repos) >= limit:
break
return repos
def _date_days_ago(days: int) -> str: def _date_days_ago(days: int) -> str:

View File

@@ -3,14 +3,17 @@
from __future__ import annotations from __future__ import annotations
import json import json
import logging
import os
import re import re
from typing import Any from typing import Any
import certifi import certifi
import httpx import httpx
from daily.config import env, env_int from daily.config import ROOT, env, env_int
from daily.cursor_client import cursor_chat
logger = logging.getLogger(__name__)
_JSON_BLOCK = re.compile(r"```(?:json)?\s*([\s\S]*?)```", re.IGNORECASE) _JSON_BLOCK = re.compile(r"```(?:json)?\s*([\s\S]*?)```", re.IGNORECASE)
@@ -41,47 +44,6 @@ def extract_json_object(text: str) -> dict[str, Any]:
return {} return {}
def _has_openai_configured() -> bool:
return bool(env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY"))
def _has_cursor_configured() -> bool:
return bool(env("CURSOR_API_KEY"))
def llm_provider() -> str:
raw = (env("DAILY_LLM_PROVIDER") or "auto").strip().lower()
if raw in {"openai", "cursor"}:
return raw
return "auto"
def resolve_llm_backend() -> str:
"""返回 openai | cursor | 空字符串。"""
provider = llm_provider()
has_openai = _has_openai_configured()
has_cursor = _has_cursor_configured()
if provider == "openai":
if has_openai:
return "openai"
return "cursor" if has_cursor else ""
if provider == "cursor":
if has_cursor:
return "cursor"
return "openai" if has_openai else ""
agent_mode = (env("DAILY_REPORT_MODE") or "").strip().lower() == "agent"
if agent_mode and has_cursor:
return "cursor"
if has_openai:
return "openai"
if has_cursor:
return "cursor"
return ""
def _openai_chat(system: str, user: str) -> str: def _openai_chat(system: str, user: str) -> str:
api_key = (env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY") or "").strip() api_key = (env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY") or "").strip()
if not api_key: if not api_key:
@@ -108,14 +70,54 @@ def _openai_chat(system: str, user: str) -> str:
return str(data["choices"][0]["message"]["content"] or "").strip() return str(data["choices"][0]["message"]["content"] or "").strip()
def _cursor_chat(system: str, user: str) -> str:
api_key = (env("CURSOR_API_KEY") or "").strip()
if not api_key:
return ""
from cursor_sdk import Agent, AgentOptions, CursorAgentError, LocalAgentOptions
from daily.bridge_manager import warm_cursor_bridge
cwd = env("DAILY_CURSOR_CWD") or str(ROOT)
os.environ["CURSOR_CWD"] = cwd
warm_cursor_bridge()
model = env("CURSOR_MODEL") or "composer-2.5"
prompt = f"{system}\n\n{user}"
try:
result = Agent.prompt(
prompt,
AgentOptions(
api_key=api_key,
model=model,
local=LocalAgentOptions(cwd=cwd),
),
)
except CursorAgentError as exc:
raise RuntimeError(f"LLM 调用失败:{exc.message}") from exc
if result.status == "error":
raise RuntimeError(f"LLM 调用失败:{result.result or '未知错误'}")
return (result.result or "").strip()
def llm_chat(system: str, user: str) -> str: def llm_chat(system: str, user: str) -> str:
backend = resolve_llm_backend() """优先 OpenAI 兼容 API否则 Cursor SDK。"""
if backend == "openai": if env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY"):
return _openai_chat(system, user) return _openai_chat(system, user)
if backend == "cursor": if env("CURSOR_API_KEY"):
return cursor_chat(system, user) return _cursor_chat(system, user)
return "" return ""
def has_cursor_configured() -> bool:
return bool((env("CURSOR_API_KEY") or "").strip())
def cursor_agent_prompt(system: str, user: str) -> str:
"""仅 Cursor SDK Agent可用 WebSearch 等工具),不走 OpenAI 兼容 API。"""
if not has_cursor_configured():
return ""
return _cursor_chat(system, user)
def has_llm_configured() -> bool: def has_llm_configured() -> bool:
return bool(resolve_llm_backend()) return bool(env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY") or env("CURSOR_API_KEY"))

149
daily/narrative_axis.py Normal file
View File

@@ -0,0 +1,149 @@
"""叙事轴硬互斥:代码选定轴,注入 Agent Step1 并强制覆写。"""
from __future__ import annotations
import json
import logging
import random
from collections import defaultdict
from datetime import datetime, timedelta
from typing import Any
from daily.config import OUTPUT_DIR, narrative_axis_days
logger = logging.getLogger(__name__)
NARRATIVE_AXES: tuple[str, ...] = (
"政策监管",
"模型发布",
"工具链/Agent",
"芯片算力",
"开源生态",
"应用落地",
"安全/诉讼",
)
def pick_narrative_axis(
used: set[str],
*,
rng: random.Random | None = None,
) -> str:
"""从固定轴枚举中排除已用轴后随机选取;全用尽则回退全表。"""
available = [a for a in NARRATIVE_AXES if a not in used]
pool = available or list(NARRATIVE_AXES)
picker = rng or random.Random()
return picker.choice(pool)
def load_recent_axes(date_str: str, days: int | None = None) -> list[str]:
"""近 N 日 data.narrative_axis不含当日按时间从近到远"""
lookback = days if days is not None else narrative_axis_days()
try:
dt = datetime.strptime(date_str, "%Y-%m-%d")
except ValueError:
return []
axes: list[str] = []
for day_offset in range(1, lookback + 1):
prev = (dt - timedelta(days=day_offset)).strftime("%Y-%m-%d")
path = OUTPUT_DIR / f"{prev}.data.json"
if not path.exists():
continue
try:
payload = json.loads(path.read_text(encoding="utf-8"))
except (OSError, ValueError) as exc:
logger.warning("读取 narrative_axis %s 失败:%s", path, exc)
continue
data = payload.get("data") if isinstance(payload, dict) else None
if not isinstance(data, dict):
continue
axis = str(data.get("narrative_axis") or "").strip()
if axis:
axes.append(axis)
return axes
def load_recent_theme_summaries(date_str: str, days: int) -> list[str]:
"""近 N 日 theme/opening 摘要,供 Step1 软禁参考。"""
try:
dt = datetime.strptime(date_str, "%Y-%m-%d")
except ValueError:
return []
summaries: list[str] = []
for day_offset in range(1, days + 1):
prev = (dt - timedelta(days=day_offset)).strftime("%Y-%m-%d")
path = OUTPUT_DIR / f"{prev}.data.json"
if not path.exists():
continue
try:
payload = json.loads(path.read_text(encoding="utf-8"))
except (OSError, ValueError):
continue
data = payload.get("data") if isinstance(payload, dict) else None
if not isinstance(data, dict):
continue
theme = str(data.get("theme") or data.get("editorial_theme") or "").strip()
opening = ""
trends = data.get("trends") if isinstance(data.get("trends"), dict) else {}
if isinstance(trends, dict):
opening = str(trends.get("opening") or "").strip()
if not theme:
themes = trends.get("themes") or []
if themes and isinstance(themes[0], dict):
theme = str(themes[0].get("title") or "").strip()
bit = " · ".join(x for x in (prev, theme, opening[:40]) if x)
if bit:
summaries.append(bit)
return summaries
def enforce_narrative_axis(trends: dict[str, Any], axis: str) -> dict[str, Any]:
"""强制 trends['narrative_axis'] = axis。"""
out = dict(trends)
out["narrative_axis"] = axis
return out
def theme_clusters(
feed: dict[str, Any],
*,
limit: int = 5,
theme_rules: list[tuple[str, str, list[str]]],
skill_id_fn: Any,
) -> list[tuple[str, list[str]]]:
"""按 THEME_RULES 把 feed 的 topTrending/topHot 聚成 (主题, 示例列表)。
从 generate.py 迁入(原私有 _theme_clusters。theme_rules 与 skill_id_fn
由调用方注入,避免对 generate.py 的反向依赖(防循环 import
"""
buckets: dict[str, list[str]] = defaultdict(list)
seen: set[str] = set()
for board in ("topTrending", "topHot"):
for item in feed.get(board, [])[:20]:
item_id = skill_id_fn(item)
if item_id in seen:
continue
seen.add(item_id)
haystack = " ".join(
[item.get("title", ""), item.get("source", ""), item.get("description", "")]
).lower()
for _icon, theme, keywords in theme_rules:
if any(k in haystack for k in keywords):
label = f"**{item.get('title')}** (`{item.get('source')}`)"
if label not in buckets[theme]:
buckets[theme].append(label)
break
return [(theme, examples[:limit]) for theme, examples in buckets.items() if examples]
def theme_names(
feed: dict[str, Any],
*,
theme_rules: list[tuple[str, str, list[str]]],
skill_id_fn: Any,
limit: int = 3,
) -> list[str]:
"""仅取主题名(不含 markdown 示例),供「今日看点/theme_line」回退文案。"""
return [theme for theme, _ in theme_clusters(
feed, theme_rules=theme_rules, skill_id_fn=skill_id_fn
)][:limit]

View File

@@ -1,13 +1,26 @@
from daily.news.fetch import ( from daily.news.fetch import (
fetch_ai_news, fetch_ai_news,
fetch_cn_ai_news, fetch_cn_ai_news,
format_cn_news_section, format_cn_news_section,
format_news_section, format_news_section,
) )
__all__ = [ __all__ = [
"fetch_ai_news", "fetch_ai_news",
"fetch_cn_ai_news", "fetch_cn_ai_news",
"format_news_section", "format_news_section",
"format_cn_news_section", "format_cn_news_section",
] ]

View File

@@ -1,77 +0,0 @@
"""敏感词内容过滤。"""
from __future__ import annotations
import logging
from functools import lru_cache
from pathlib import Path
from typing import Any
from daily.config import ROOT, env
logger = logging.getLogger(__name__)
_WORDS_FILE = ROOT / "config" / "sensitive_words.yaml"
def content_filter_enabled() -> bool:
raw = (env("DAILY_CONTENT_FILTER") or "0").strip().lower()
return raw in {"1", "true", "yes", "on"}
@lru_cache(maxsize=1)
def load_sensitive_words() -> tuple[str, ...]:
if not _WORDS_FILE.exists():
return ()
try:
import yaml
except ImportError:
logger.warning("未安装 PyYAML无法读取 %s", _WORDS_FILE)
return ()
try:
data = yaml.safe_load(_WORDS_FILE.read_text(encoding="utf-8"))
except OSError as exc:
logger.warning("读取敏感词配置失败: %s", exc)
return ()
except Exception as exc:
logger.warning("解析 sensitive_words.yaml 失败: %s", exc)
return ()
if not isinstance(data, dict):
return ()
words = data.get("words") or data.get("sensitive_words") or []
if not isinstance(words, list):
return ()
cleaned = tuple(str(word).strip() for word in words if str(word).strip())
return cleaned
def matches_sensitive_text(text: str, words: tuple[str, ...]) -> str | None:
haystack = (text or "").lower()
if not haystack:
return None
for word in words:
needle = word.lower()
if needle and needle in haystack:
return word
return None
def filter_news_items(items: list[dict[str, Any]]) -> tuple[list[dict[str, Any]], int]:
if not content_filter_enabled():
return items, 0
words = load_sensitive_words()
if not words:
return items, 0
kept: list[dict[str, Any]] = []
removed = 0
for item in items:
text = f"{item.get('title', '')} {item.get('summary', '')}"
hit = matches_sensitive_text(text, words)
if hit:
removed += 1
continue
kept.append(item)
if removed:
logger.info("内容过滤移除 %d 条(敏感词)", removed)
return kept, removed

View File

@@ -1,10 +1,125 @@
"""国际 AI 时讯 RSS 源(优先 config/feeds.yaml)。""" """国际 AI 时讯 RSS 源定义(按类别分组)。"""
from __future__ import annotations from __future__ import annotations
from daily.news.feeds_loader import load_categories from dataclasses import dataclass
from daily.news.feeds_types import NewsCategory, NewsFeed
NEWS_CATEGORIES: tuple[NewsCategory, ...] = load_categories("intl")
__all__ = ["NewsCategory", "NewsFeed", "NEWS_CATEGORIES"] @dataclass(frozen=True)
class NewsFeed:
name: str
url: str
slow: bool = False # 限速源(如 Reddit串行抓取
ai_filter: bool = False # 综合源仅保留标题命中 AI 关键词的条目
@dataclass(frozen=True)
class NewsCategory:
id: str
name: str
icon: str
feeds: tuple[NewsFeed, ...]
NEWS_CATEGORIES: tuple[NewsCategory, ...] = (
NewsCategory(
id="official",
name="厂商官方",
icon="🏢",
feeds=(
NewsFeed("Anthropic Claude 更新", "https://platform.claude.com/docs/en/release-notes/overview"),
NewsFeed("OpenAI", "https://openai.com/news/rss.xml"),
NewsFeed("Google AI", "https://blog.google/technology/ai/rss/"),
NewsFeed("DeepMind", "https://deepmind.google/blog/rss.xml"),
NewsFeed("Meta Engineering", "https://engineering.fb.com/feed/"),
NewsFeed("Microsoft Research", "https://www.microsoft.com/en-us/research/feed/"),
NewsFeed("Microsoft Blog", "https://blogs.microsoft.com/feed/"),
NewsFeed("Cohere", "https://cohere.com/blog/rss.xml"),
NewsFeed("Cursor Changelog", "https://cursor.com/changelog/rss.xml"),
),
),
NewsCategory(
id="developer",
name="Agent / LLM 开发者",
icon="🛠",
feeds=(
NewsFeed("LangChain", "https://blog.langchain.dev/rss/"),
NewsFeed("Hugging Face", "https://huggingface.co/blog/feed.xml"),
NewsFeed("Vercel Changelog", "https://vercel.com/changelog/rss.xml"),
NewsFeed("GitHub Copilot", "https://github.blog/changelog/label/copilot/feed/"),
),
),
NewsCategory(
id="media",
name="综合科技媒体",
icon="📰",
feeds=(
NewsFeed("The Verge AI", "https://www.theverge.com/rss/ai-artificial-intelligence/index.xml"),
NewsFeed("TechCrunch AI", "https://techcrunch.com/category/artificial-intelligence/feed/"),
NewsFeed("Ars Technica AI", "https://arstechnica.com/ai/feed/"),
NewsFeed("Wired AI", "https://www.wired.com/feed/tag/ai/latest/rss"),
NewsFeed("MIT Tech Review", "https://www.technologyreview.com/feed/"),
NewsFeed("VentureBeat AI", "https://venturebeat.com/category/ai/feed/"),
),
),
NewsCategory(
id="newsletter",
name="Newsletter 日报",
icon="✉️",
feeds=(
NewsFeed("Ben's Bites", "https://bensbites.substack.com/feed"),
NewsFeed("The Rundown AI", "https://therundown.substack.com/feed"),
NewsFeed("Latent Space", "https://www.latent.space/feed"),
NewsFeed("Simon Willison", "https://simonwillison.net/atom/everything/"),
NewsFeed("Import AI", "https://importai.substack.com/feed"),
NewsFeed("Last Week in AI", "https://lastweekin.ai/feed"),
NewsFeed("The Neuron", "https://www.theneuron.ai/feed"),
),
),
NewsCategory(
id="research",
name="研究 / 论文",
icon="📚",
feeds=(
NewsFeed("arXiv cs.CL", "https://arxiv.org/rss/cs.CL"),
NewsFeed("arXiv cs.AI", "https://arxiv.org/rss/cs.AI"),
NewsFeed("arXiv cs.LG", "https://arxiv.org/rss/cs.LG"),
),
),
NewsCategory(
id="trending",
name="热点 / 趋势",
icon="🔥",
feeds=(
NewsFeed(
"Google News · AI",
"https://news.google.com/rss/search?q=artificial+intelligence+OR+LLM+OR+Claude+OR+GPT&hl=en-US&gl=US&ceid=US:en",
),
NewsFeed(
"Google News · Technology",
"https://news.google.com/rss/headlines/section/topic/TECHNOLOGY?hl=en-US&gl=US&ceid=US:en",
),
NewsFeed("Techmeme", "https://www.techmeme.com/feed.xml"),
NewsFeed("HN · Front Page", "https://hnrss.org/frontpage"),
NewsFeed("HN · 100+ Points", "https://hnrss.org/newest?points=100"),
NewsFeed("Dev.to · AI", "https://dev.to/feed/tag/ai"),
NewsFeed("Lobsters", "https://lobste.rs/rss"),
),
),
NewsCategory(
id="community",
name="社区讨论",
icon="💬",
feeds=(
NewsFeed(
"HN · AI/LLM/Agent",
"https://hnrss.org/newest?q=AI+OR+LLM+OR+Claude+OR+agent+OR+GPT+OR+Gemini",
),
NewsFeed(
"Reddit · LLM/Claude/ML",
"https://old.reddit.com/r/LocalLLaMA+ClaudeAI+MachineLearning+OpenAI/.rss?limit=25",
slow=True,
),
),
),
)

View File

@@ -1,11 +1,62 @@
"""国内 AI 时讯 RSS 源(优先 config/feeds.yaml)。""" """国内 AI 时讯 RSS 源定义(按类别分组)。"""
from __future__ import annotations from __future__ import annotations
from daily.news.feeds_loader import load_categories, load_cn_title_keywords from daily.news.feeds import NewsCategory, NewsFeed
from daily.news.feeds_types import NewsCategory, NewsFeed
CN_AI_TITLE_KEYWORDS: tuple[str, ...] = load_cn_title_keywords() # 综合源 ai_filter=True 时,仅保留标题命中以下词之一的条目
CN_NEWS_CATEGORIES: tuple[NewsCategory, ...] = load_categories("cn") CN_AI_TITLE_KEYWORDS: tuple[str, ...] = (
"人工智能",
"大模型",
"智能体",
"多模态",
"AIGC",
"LLM",
"GPT",
"Claude",
"Gemini",
"ChatGPT",
"OpenAI",
"Anthropic",
"Copilot",
"Agent",
"AI ",
" AI",
"AI·",
"AI业务",
"AI模型",
"AI助手",
"AI工具",
"AI编程",
"AI 编程",
"AI版",
"AI Agent",
"推理模型",
"深度学习",
"机器学习",
"Function Calling",
)
__all__ = ["CN_AI_TITLE_KEYWORDS", "CN_NEWS_CATEGORIES", "NewsCategory", "NewsFeed"] CN_NEWS_CATEGORIES: tuple[NewsCategory, ...] = (
NewsCategory(
id="media",
name="AI 专业媒体",
icon="📰",
feeds=(
NewsFeed("量子位", "https://www.qbitai.com/feed"),
),
),
NewsCategory(
id="tech",
name="综合科技",
icon="📱",
feeds=(
NewsFeed("36氪", "https://36kr.com/feed", ai_filter=True),
NewsFeed("雷锋网", "https://www.leiphone.com/feed"),
NewsFeed(
"Google News · AI",
"https://news.google.com/rss/search?q=人工智能+OR+大模型+OR+Agent+OR+LLM&hl=zh-CN&gl=CN&ceid=CN:zh-Hans",
),
),
),
)

View File

@@ -1,11 +0,0 @@
"""Built-in RSS defaults when config/feeds.yaml is missing or invalid."""
from __future__ import annotations
from daily.news.feeds_types import NewsCategory, NewsFeed
CN_AI_TITLE_KEYWORDS: tuple[str, ...] = ('人工智能', '大模型', '智能体', '多模态', 'AIGC', 'LLM', 'GPT', 'Claude', 'Gemini', 'ChatGPT', 'OpenAI', 'Anthropic', 'Copilot', 'Agent', 'AI ', ' AI', 'AI·', 'AI业务', 'AI模型', 'AI助手', 'AI工具', 'AI编程', 'AI 编程', 'AI版', 'AI Agent', '推理模型', '深度学习', '机器学习', 'Function Calling')
CN_NEWS_CATEGORIES: tuple[NewsCategory, ...] = (NewsCategory(id='media', name='AI 专业媒体', icon='📰', feeds=(NewsFeed(name='量子位', url='https://www.qbitai.com/feed', slow=False, ai_filter=False), NewsFeed(name='InfoQ 中文', url='https://www.infoq.cn/feed/AI', slow=False, ai_filter=False))), NewsCategory(id='tech', name='综合科技', icon='📱', feeds=(NewsFeed(name='36氪', url='https://36kr.com/feed', slow=False, ai_filter=True), NewsFeed(name='雷锋网', url='https://www.leiphone.com/feed', slow=False, ai_filter=False), NewsFeed(name='Google News · AI', url='https://news.google.com/rss/search?q=人工智能+OR+大模型+OR+Agent+OR+LLM&hl=zh-CN&gl=CN&ceid=CN:zh-Hans', slow=False, ai_filter=False))), NewsCategory(id='dev', name='开发者社区', icon='💻', feeds=(NewsFeed(name='掘金', url='https://juejin.cn/rss', slow=False, ai_filter=True),)))
NEWS_CATEGORIES: tuple[NewsCategory, ...] = (NewsCategory(id='official', name='厂商官方', icon='🏢', feeds=(NewsFeed(name='Anthropic Claude 更新', url='https://docs.anthropic.com/en/release-notes/feed', slow=False, ai_filter=False), NewsFeed(name='OpenAI', url='https://openai.com/news/rss.xml', slow=False, ai_filter=False), NewsFeed(name='Google AI', url='https://blog.google/technology/ai/rss/', slow=False, ai_filter=False), NewsFeed(name='DeepMind', url='https://deepmind.google/blog/rss.xml', slow=False, ai_filter=False), NewsFeed(name='Meta Engineering', url='https://engineering.fb.com/feed/', slow=False, ai_filter=False), NewsFeed(name='Microsoft Research', url='https://www.microsoft.com/en-us/research/feed/', slow=False, ai_filter=False), NewsFeed(name='Microsoft Blog', url='https://blogs.microsoft.com/feed/', slow=False, ai_filter=False), NewsFeed(name='Cohere', url='https://cohere.com/blog/rss.xml', slow=False, ai_filter=False), NewsFeed(name='Cursor Changelog', url='https://cursor.com/changelog/rss.xml', slow=False, ai_filter=False))), NewsCategory(id='developer', name='Agent / LLM 开发者', icon='🛠', feeds=(NewsFeed(name='LangChain', url='https://blog.langchain.dev/rss/', slow=False, ai_filter=False), NewsFeed(name='Hugging Face', url='https://huggingface.co/blog/feed.xml', slow=False, ai_filter=False), NewsFeed(name='Vercel Changelog', url='https://vercel.com/changelog/rss.xml', slow=False, ai_filter=False), NewsFeed(name='GitHub Copilot', url='https://github.blog/changelog/label/copilot/feed/', slow=False, ai_filter=False))), NewsCategory(id='media', name='综合科技媒体', icon='📰', feeds=(NewsFeed(name='The Verge AI', url='https://www.theverge.com/rss/ai-artificial-intelligence/index.xml', slow=False, ai_filter=False), NewsFeed(name='TechCrunch AI', url='https://techcrunch.com/category/artificial-intelligence/feed/', slow=False, ai_filter=False), NewsFeed(name='Ars Technica AI', url='https://arstechnica.com/ai/feed/', slow=False, ai_filter=False), NewsFeed(name='Wired AI', url='https://www.wired.com/feed/tag/ai/latest/rss', slow=False, ai_filter=False), NewsFeed(name='MIT Tech Review', url='https://www.technologyreview.com/feed/', slow=False, ai_filter=False), NewsFeed(name='VentureBeat AI', url='https://venturebeat.com/category/ai/feed/', slow=False, ai_filter=False))), NewsCategory(id='newsletter', name='Newsletter 日报', icon='✉️', feeds=(NewsFeed(name="Ben's Bites", url='https://bensbites.substack.com/feed', slow=False, ai_filter=False), NewsFeed(name='The Rundown AI', url='https://therundown.substack.com/feed', slow=False, ai_filter=False), NewsFeed(name='Latent Space', url='https://www.latent.space/feed', slow=False, ai_filter=False), NewsFeed(name='Simon Willison', url='https://simonwillison.net/atom/everything/', slow=False, ai_filter=False), NewsFeed(name='Import AI', url='https://importai.substack.com/feed', slow=False, ai_filter=False), NewsFeed(name='Last Week in AI', url='https://lastweekin.ai/feed', slow=False, ai_filter=False), NewsFeed(name='The Neuron', url='https://www.theneuron.ai/feed', slow=False, ai_filter=False))), NewsCategory(id='research', name='研究 / 论文', icon='📚', feeds=(NewsFeed(name='arXiv cs.CL', url='https://arxiv.org/rss/cs.CL', slow=False, ai_filter=False), NewsFeed(name='arXiv cs.AI', url='https://arxiv.org/rss/cs.AI', slow=False, ai_filter=False), NewsFeed(name='arXiv cs.LG', url='https://arxiv.org/rss/cs.LG', slow=False, ai_filter=False))), NewsCategory(id='trending', name='热点 / 趋势', icon='🔥', feeds=(NewsFeed(name='Google News · AI', url='https://news.google.com/rss/search?q=artificial+intelligence+OR+LLM+OR+Claude+OR+GPT&hl=en-US&gl=US&ceid=US:en', slow=False, ai_filter=False), NewsFeed(name='Google News · Technology', url='https://news.google.com/rss/headlines/section/topic/TECHNOLOGY?hl=en-US&gl=US&ceid=US:en', slow=False, ai_filter=False), NewsFeed(name='Techmeme', url='https://www.techmeme.com/feed.xml', slow=False, ai_filter=False), NewsFeed(name='HN · Front Page', url='https://hnrss.org/frontpage', slow=False, ai_filter=False), NewsFeed(name='HN · 100+ Points', url='https://hnrss.org/newest?points=100', slow=False, ai_filter=False), NewsFeed(name='Dev.to · AI', url='https://dev.to/feed/tag/ai', slow=False, ai_filter=False), NewsFeed(name='Lobsters', url='https://lobste.rs/rss', slow=False, ai_filter=False))), NewsCategory(id='community', name='社区讨论', icon='💬', feeds=(NewsFeed(name='HN · AI/LLM/Agent', url='https://hnrss.org/newest?q=AI+OR+LLM+OR+Claude+OR+agent+OR+GPT+OR+Gemini', slow=False, ai_filter=False), NewsFeed(name='Reddit · LLM/Claude/ML', url='https://old.reddit.com/r/LocalLLaMA+ClaudeAI+MachineLearning+OpenAI/.rss?limit=25', slow=True, ai_filter=False))))

View File

@@ -1,113 +0,0 @@
"""从 config/feeds.yaml 加载 RSS 源(失败时回退内置默认)。"""
from __future__ import annotations
import logging
from functools import lru_cache
from pathlib import Path
from typing import Any
from daily.config import ROOT
from daily.news.feeds_types import NewsCategory, NewsFeed
logger = logging.getLogger(__name__)
_FEEDS_FILE = ROOT / "config" / "feeds.yaml"
def _parse_feed(raw: dict[str, Any]) -> NewsFeed:
return NewsFeed(
name=str(raw.get("name") or "").strip(),
url=str(raw.get("url") or "").strip(),
slow=bool(raw.get("slow")),
ai_filter=bool(raw.get("ai_filter")),
)
def _parse_category(raw: dict[str, Any]) -> NewsCategory | None:
cat_id = str(raw.get("id") or "").strip()
if not cat_id:
return None
feeds_raw = raw.get("feeds") or []
feeds: list[NewsFeed] = []
for item in feeds_raw:
if not isinstance(item, dict):
continue
feed = _parse_feed(item)
if feed.name and feed.url:
feeds.append(feed)
if not feeds:
return None
return NewsCategory(
id=cat_id,
name=str(raw.get("name") or cat_id),
icon=str(raw.get("icon") or "📰"),
feeds=tuple(feeds),
)
def _parse_categories(items: Any) -> tuple[NewsCategory, ...]:
if not isinstance(items, list):
return ()
categories: list[NewsCategory] = []
for raw in items:
if not isinstance(raw, dict):
continue
cat = _parse_category(raw)
if cat:
categories.append(cat)
return tuple(categories)
def _load_yaml() -> dict[str, Any] | None:
if not _FEEDS_FILE.exists():
return None
try:
import yaml
except ImportError:
logger.warning("未安装 PyYAML无法读取 %s", _FEEDS_FILE)
return None
try:
data = yaml.safe_load(_FEEDS_FILE.read_text(encoding="utf-8"))
except OSError as exc:
logger.warning("读取 feeds 配置失败: %s", exc)
return None
except Exception as exc:
logger.warning("解析 feeds.yaml 失败: %s", exc)
return None
return data if isinstance(data, dict) else None
@lru_cache(maxsize=1)
def _yaml_payload() -> dict[str, Any] | None:
return _load_yaml()
def load_categories(region: str) -> tuple[NewsCategory, ...]:
data = _yaml_payload()
if data:
block = data.get(region) or {}
categories = _parse_categories(block.get("categories"))
if categories:
return categories
logger.warning("feeds.yaml 中 %s.categories 为空,使用内置默认", region)
from daily.news import feeds_defaults as defaults
if region == "cn":
return defaults.CN_NEWS_CATEGORIES
return defaults.NEWS_CATEGORIES
def load_cn_title_keywords() -> tuple[str, ...]:
data = _yaml_payload()
if data:
block = data.get("cn") or {}
keywords = block.get("title_keywords")
if isinstance(keywords, list):
cleaned = tuple(str(x).strip() for x in keywords if str(x).strip())
if cleaned:
return cleaned
from daily.news.feeds_defaults import CN_AI_TITLE_KEYWORDS
return CN_AI_TITLE_KEYWORDS

View File

@@ -1,21 +0,0 @@
"""RSS 源数据结构。"""
from __future__ import annotations
from dataclasses import dataclass
@dataclass(frozen=True)
class NewsFeed:
name: str
url: str
slow: bool = False
ai_filter: bool = False
@dataclass(frozen=True)
class NewsCategory:
id: str
name: str
icon: str
feeds: tuple[NewsFeed, ...]

View File

@@ -8,28 +8,28 @@ import time
import html import html
import xml.etree.ElementTree as ET import xml.etree.ElementTree as ET
from concurrent.futures import ThreadPoolExecutor, as_completed from concurrent.futures import ThreadPoolExecutor, as_completed
from datetime import datetime, timezone, timedelta from datetime import datetime, time as dt_time, timezone, timedelta
from email.utils import parsedate_to_datetime from email.utils import parsedate_to_datetime
from typing import Any from typing import Any
from urllib.parse import parse_qs, urlencode, urlparse, urlunparse from urllib.parse import parse_qs, urlencode, urlparse, urlunparse
from zoneinfo import ZoneInfo
import certifi import certifi
import httpx import httpx
from daily.config import env, env_int, news_summary_limit from daily.config import env, env_int, news_summary_limit, wecom_news_desc_limit
from daily.news.content_filter import filter_news_items
from daily.news.feeds import NEWS_CATEGORIES, NewsCategory, NewsFeed from daily.news.feeds import NEWS_CATEGORIES, NewsCategory, NewsFeed
from daily.news.feeds_cn import CN_AI_TITLE_KEYWORDS, CN_NEWS_CATEGORIES from daily.news.feeds_cn import CN_AI_TITLE_KEYWORDS, CN_NEWS_CATEGORIES
from daily.text_utils import trim_brief
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
# 部分站点(如 InfoQ会拦截含 bot 标识的 UARSS 抓取统一用浏览器 UA USER_AGENT = "Mozilla/5.0 (compatible; skills-hot-daily/1.0; +https://skills.sh)"
RSS_USER_AGENT = ( BROWSER_USER_AGENT = (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) " "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) " "AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/131.0.0.0 Safari/537.36" "Chrome/131.0.0.0 Safari/537.36"
) )
BROWSER_USER_AGENT = RSS_USER_AGENT
STRIP_HTML = re.compile(r"<[^>]+>") STRIP_HTML = re.compile(r"<[^>]+>")
WS = re.compile(r"\s+") WS = re.compile(r"\s+")
@@ -48,15 +48,40 @@ def _cn_enabled() -> bool:
def _hours_window() -> int: def _hours_window() -> int:
return max(1, env_int("DAILY_AI_NEWS_HOURS", 72)) return max(1, env_int("DAILY_AI_NEWS_HOURS", 24))
def _news_tz_name() -> str:
return (env("DAILY_AI_NEWS_TZ") or env("DAILY_SCHEDULE_TZ") or "Asia/Shanghai").strip()
def _floor_today_enabled() -> bool:
raw = env("DAILY_AI_NEWS_FLOOR_TODAY")
if raw is None:
return True
return raw.strip().lower() not in {"0", "false", "no", "off"}
def _cutoff_datetime(*, floor_today: bool) -> datetime:
"""滚动 N 小时窗口;国际新闻可叠加「不早于今日 0 点(本地时区)」。"""
now = _now_utc()
rolling = now - timedelta(hours=_hours_window())
if not floor_today:
return rolling
tz = ZoneInfo(_news_tz_name())
local = now.astimezone(tz)
start_today = local.replace(hour=0, minute=0, second=0, microsecond=0).astimezone(timezone.utc)
return max(rolling, start_today)
def _per_feed_limit() -> int: def _per_feed_limit() -> int:
return max(1, env_int("DAILY_AI_NEWS_PER_FEED", 3)) want = max(_wecom_limit(), _wecom_cn_limit())
return max(want // 2, env_int("DAILY_AI_NEWS_PER_FEED", 5))
def _per_category_limit() -> int: def _per_category_limit() -> int:
return max(1, env_int("DAILY_AI_NEWS_PER_CATEGORY", 5)) want = max(_wecom_limit(), _wecom_cn_limit())
return max(want, env_int("DAILY_AI_NEWS_PER_CATEGORY", 10))
def _wecom_limit() -> int: def _wecom_limit() -> int:
@@ -64,7 +89,7 @@ def _wecom_limit() -> int:
def _wecom_cn_limit() -> int: def _wecom_cn_limit() -> int:
return max(1, env_int("DAILY_WECOM_CN_AI_NEWS", 8)) return max(1, env_int("DAILY_WECOM_CN_AI_NEWS", 10))
def _matches_cn_ai_title(title: str) -> bool: def _matches_cn_ai_title(title: str) -> bool:
@@ -104,7 +129,6 @@ def _parse_datetime(value: str | None) -> datetime | None:
for fmt in ( for fmt in (
"%Y-%m-%dT%H:%M:%SZ", "%Y-%m-%dT%H:%M:%SZ",
"%Y-%m-%dT%H:%M:%S%z", "%Y-%m-%dT%H:%M:%S%z",
"%Y-%m-%d",
): ):
try: try:
dt = datetime.strptime(text[: len(fmt.replace("%z", "+0000"))], fmt.replace("%z", "")) dt = datetime.strptime(text[: len(fmt.replace("%z", "+0000"))], fmt.replace("%z", ""))
@@ -113,17 +137,103 @@ def _parse_datetime(value: str | None) -> datetime | None:
return dt.astimezone(timezone.utc) return dt.astimezone(timezone.utc)
except ValueError: except ValueError:
continue continue
if re.match(r"^\d{4}-\d{2}-\d{2}$", text):
try:
tz = ZoneInfo(_news_tz_name())
day = datetime.strptime(text, "%Y-%m-%d").date()
# 仅日期时按本地中午估算,避免 UTC 0 点误判为「前一天」
dt = datetime.combine(day, dt_time(12, 0), tzinfo=tz)
return dt.astimezone(timezone.utc)
except ValueError:
pass
return None return None
def _clean_text(text: str | None, limit: int = 200) -> str: def _clean_text(text: str | None, limit: int = 200) -> str:
if not text: if not text:
return "" return ""
plain = STRIP_HTML.sub(" ", html.unescape(text)) plain = _strip_summary_plain(text)
plain = WS.sub(" ", plain).strip()
if limit <= 0 or len(plain) <= limit: if limit <= 0 or len(plain) <= limit:
return plain return plain
return plain[: limit - 3] + "..." return trim_brief(plain, limit)
def _strip_summary_plain(text: str | None) -> str:
if not text:
return ""
plain = STRIP_HTML.sub(" ", html.unescape(text))
return WS.sub(" ", plain).strip()
_JUNK_SUMMARY_RE = re.compile(
r"^(点击查看原文|article url:|comments url:|discussion on hn|read more)",
re.IGNORECASE,
)
def _is_junk_news_summary(text: str) -> bool:
if not text:
return True
if _JUNK_SUMMARY_RE.match(text.strip()):
return True
if text.strip().endswith(">") and "点击" in text:
return True
return False
def brief_news_summary(text: str | None, limit: int | None = None) -> str:
"""企微新闻一句摘要:去 HTML、过滤占位文案、句读处截断。"""
plain = _strip_summary_plain(text)
if _is_junk_news_summary(plain):
return ""
lim = wecom_news_desc_limit() if limit is None else limit
return trim_brief(plain, lim)
def sync_wecom_news_rows(items: list[dict[str, Any]], flat: list[dict[str, Any]]) -> None:
"""中文化后,用 flat 最新 summary 刷新企微 desc_short。"""
by_link = {_normalize_link(str(i.get("link") or "")): i for i in flat if i.get("link")}
for row in items:
link = _normalize_link(str(row.get("link") or ""))
src = by_link.get(link)
if src:
row["desc_short"] = brief_news_summary(src.get("summary"))
def finalize_wecom_news_items(
items: list[dict[str, Any]],
*,
force_chinese: bool = False,
) -> None:
"""企微新闻摘要:确保 desc_short 为中文(国际源 force_chinese=True"""
from daily.localize import LocalizeJob, localize_brief_descriptions, needs_chinese
from daily.news.sanitize import strip_relax_window_prefix
limit = wecom_news_desc_limit()
jobs: list[LocalizeJob] = []
keyed: list[tuple[str, dict[str, Any]]] = []
for idx, item in enumerate(items):
text = strip_relax_window_prefix(
(item.get("desc_short") or item.get("summary_plain") or "").strip()
)
if text:
item["desc_short"] = text
if not text or _is_junk_news_summary(text):
item["desc_short"] = ""
continue
if force_chinese or needs_chinese(text):
key = f"wecom-news:{item.get('link') or idx}"
jobs.append(LocalizeJob(key, text, limit))
keyed.append((key, item))
elif not item.get("desc_short"):
item["desc_short"] = brief_news_summary(text, limit)
if not jobs:
return
zh_map = localize_brief_descriptions(jobs, archive=True)
for key, item in keyed:
if key in zh_map:
item["desc_short"] = strip_relax_window_prefix(zh_map[key])
def _normalize_link(link: str) -> str: def _normalize_link(link: str) -> str:
@@ -288,25 +398,20 @@ def _fetch_one(
client: httpx.Client, client: httpx.Client,
category: NewsCategory, category: NewsCategory,
feed: NewsFeed, feed: NewsFeed,
) -> tuple[list[dict[str, Any]], bool]: ) -> list[dict[str, Any]]:
max_attempts = max(1, env_int("DAILY_RSS_RETRY", 3))
last_exc: Exception | None = None last_exc: Exception | None = None
for url in _reddit_fetch_urls(feed.url): for url in _reddit_fetch_urls(feed.url):
for attempt in range(max_attempts): try:
try: headers = _request_headers(dict(client.headers), url)
headers = _request_headers(dict(client.headers), url) resp = client.get(url, headers=headers)
resp = client.get(url, headers=headers) resp.raise_for_status()
resp.raise_for_status() entries = _parse_feed(resp.text, feed.name, category)
entries = _parse_feed(resp.text, feed.name, category) return _filter_ai_entries(entries, ai_filter=feed.ai_filter)
return _filter_ai_entries(entries, ai_filter=feed.ai_filter), True except Exception as exc:
except Exception as exc: last_exc = exc
last_exc = exc continue
if attempt + 1 < max_attempts:
time.sleep(min(2.0 * (attempt + 1), 5.0))
continue
break
logger.warning("RSS fetch failed [%s] %s: %s", feed.name, feed.url, last_exc) logger.warning("RSS fetch failed [%s] %s: %s", feed.name, feed.url, last_exc)
return [], False return []
def _dedupe_items(items: list[dict[str, Any]]) -> list[dict[str, Any]]: def _dedupe_items(items: list[dict[str, Any]]) -> list[dict[str, Any]]:
@@ -324,10 +429,119 @@ def _dedupe_items(items: list[dict[str, Any]]) -> list[dict[str, Any]]:
return result return result
def _filter_flat_in_window(
flat: list[dict[str, Any]],
*,
floor_today: bool,
) -> list[dict[str, Any]]:
cutoff = _cutoff_datetime(floor_today=floor_today)
items = [i for i in _dedupe_items(flat) if _within_window(i, cutoff)]
items.sort(key=_sort_key, reverse=True)
return items
def _pick_news_items(
flat: list[dict[str, Any]],
limit: int,
preferred: tuple[str, ...],
*,
one_per_source: bool = False,
) -> list[dict[str, Any]]:
picked: list[dict[str, Any]] = []
seen_links: set[str] = set()
seen_sources: set[str] = set()
def _try_take(item: dict[str, Any]) -> bool:
link = _normalize_link(item.get("link", ""))
if not link or link in seen_links:
return False
if one_per_source:
source = item.get("source_name", "?")
if source in seen_sources:
return False
seen_sources.add(source)
seen_links.add(link)
picked.append(item)
return True
for cat in preferred:
for item in flat:
if item.get("category_id") != cat:
continue
if _try_take(item) and len(picked) >= limit:
return picked[:limit]
for item in flat:
if _try_take(item) and len(picked) >= limit:
break
return picked[:limit]
def _fill_picked_to_limit(
picked: list[dict[str, Any]],
pools: list[list[dict[str, Any]]],
limit: int,
) -> list[dict[str, Any]]:
seen_links = {_normalize_link(i.get("link", "")) for i in picked}
for pool in pools:
for item in pool:
if len(picked) >= limit:
return picked[:limit]
link = _normalize_link(item.get("link", ""))
if not link or link in seen_links:
continue
picked.append(item)
seen_links.add(link)
return picked[:limit]
def _to_wecom_news_row(item: dict[str, Any]) -> dict[str, Any]:
plain = _strip_summary_plain(item.get("summary", ""))
return {
"title": item.get("title", "?"),
"link": item.get("link", ""),
"source_name": item.get("source_name", "?"),
"published_fmt": item.get("published_fmt", ""),
"desc_short": brief_news_summary(plain),
"summary_plain": plain,
}
def _apply_pushed_dedup_with_backfill(
items: list[dict[str, Any]],
picked: list[dict[str, Any]],
*,
date_str: str | None,
limit: int,
) -> list[dict[str, Any]]:
if not date_str:
return items[:limit]
from daily.config import news_backfill_enabled
from daily.news.pushed_links import filter_unpushed_items
fresh = filter_unpushed_items(items, date_str=date_str)
if len(fresh) >= limit:
return fresh[:limit]
if not news_backfill_enabled():
if len(fresh) < limit:
logger.info("news_short:%s", len(fresh))
return fresh[:limit]
seen = {_normalize_link(i.get("link", "")) for i in fresh if i.get("link")}
for item in picked:
if len(fresh) >= limit:
break
link = _normalize_link(item.get("link", ""))
if not link or link in seen:
continue
fresh.append(_to_wecom_news_row(item))
seen.add(link)
return fresh[:limit]
def _within_window(item: dict[str, Any], cutoff: datetime) -> bool: def _within_window(item: dict[str, Any], cutoff: datetime) -> bool:
dt = _entry_datetime(item) dt = _entry_datetime(item)
if dt is None: if dt is None:
return True return False
return dt >= cutoff return dt >= cutoff
@@ -338,26 +552,20 @@ def _sort_key(item: dict[str, Any]) -> tuple[int, datetime]:
return (0, dt) return (0, dt)
def _fetch_news(categories: tuple[NewsCategory, ...]) -> dict[str, Any]: def _fetch_news(categories: tuple[NewsCategory, ...], *, floor_today: bool = False) -> dict[str, Any]:
hours = _hours_window() hours = _hours_window()
per_feed = _per_feed_limit() per_feed = _per_feed_limit()
per_category = _per_category_limit() per_category = _per_category_limit()
cutoff = _now_utc() - timedelta(hours=hours) cutoff = _cutoff_datetime(floor_today=floor_today)
headers = {"User-Agent": RSS_USER_AGENT, "Accept": "application/rss+xml, application/atom+xml, application/xml, text/xml, */*"} headers = {"User-Agent": USER_AGENT, "Accept": "application/rss+xml, application/atom+xml, application/xml, text/xml, */*"}
tasks: list[tuple[NewsCategory, NewsFeed]] = [] tasks: list[tuple[NewsCategory, NewsFeed]] = []
for category in categories: for category in categories:
for feed in category.feeds: for feed in category.feeds:
tasks.append((category, feed)) tasks.append((category, feed))
raw_by_category: dict[str, list[dict[str, Any]]] = {c.id: [] for c in categories} raw_by_category: dict[str, list[dict[str, Any]]] = {c.id: [] for c in categories}
stats: dict[str, Any] = { stats = {"feeds_total": len(tasks), "feeds_ok": 0, "items_raw": 0}
"feeds_total": len(tasks),
"feeds_ok": 0,
"items_raw": 0,
"feeds_failed": [],
"content_filtered": 0,
}
with httpx.Client(timeout=15.0, verify=certifi.where(), follow_redirects=True, headers=headers) as client: with httpx.Client(timeout=15.0, verify=certifi.where(), follow_redirects=True, headers=headers) as client:
fast_tasks = [t for t in tasks if not t[1].slow] fast_tasks = [t for t in tasks if not t[1].slow]
@@ -371,24 +579,19 @@ def _fetch_news(categories: tuple[NewsCategory, ...]) -> dict[str, Any]:
for future in as_completed(futures): for future in as_completed(futures):
cat_id, feed_name = futures[future] cat_id, feed_name = futures[future]
try: try:
entries, ok = future.result() entries = future.result()
except Exception as exc: except Exception as exc:
logger.warning("RSS 任务异常 [%s]: %s", feed_name, exc) logger.warning("RSS 任务异常 [%s]: %s", feed_name, exc)
stats["feeds_failed"].append(feed_name)
continue continue
if ok: if entries:
stats["feeds_ok"] += 1 stats["feeds_ok"] += 1
else:
stats["feeds_failed"].append(feed_name)
stats["items_raw"] += len(entries) stats["items_raw"] += len(entries)
raw_by_category[cat_id].extend(entries[:per_feed]) raw_by_category[cat_id].extend(entries[:per_feed])
for cat, feed in slow_tasks: for cat, feed in slow_tasks:
entries, ok = _fetch_one(client, cat, feed) entries = _fetch_one(client, cat, feed)
if ok: if entries:
stats["feeds_ok"] += 1 stats["feeds_ok"] += 1
else:
stats["feeds_failed"].append(feed.name)
stats["items_raw"] += len(entries) stats["items_raw"] += len(entries)
raw_by_category[cat.id].extend(entries[:per_feed]) raw_by_category[cat.id].extend(entries[:per_feed])
if _is_reddit_url(feed.url): if _is_reddit_url(feed.url):
@@ -402,9 +605,6 @@ def _fetch_news(categories: tuple[NewsCategory, ...]) -> dict[str, Any]:
for category in categories: for category in categories:
items = raw_by_category[category.id] items = raw_by_category[category.id]
items = [i for i in items if _within_window(i, cutoff)] items = [i for i in items if _within_window(i, cutoff)]
items, filtered_count = filter_news_items(items)
if filtered_count:
stats["content_filtered"] = stats.get("content_filtered", 0) + filtered_count
items.sort(key=_sort_key, reverse=True) items.sort(key=_sort_key, reverse=True)
items = _dedupe_items(items)[:per_category] items = _dedupe_items(items)[:per_category]
for item in items: for item in items:
@@ -427,6 +627,7 @@ def _fetch_news(categories: tuple[NewsCategory, ...]) -> dict[str, Any]:
return { return {
"enabled": True, "enabled": True,
"hours": hours, "hours": hours,
"floor_today": floor_today,
"categories": categories_out, "categories": categories_out,
"flat": flat, "flat": flat,
"stats": stats, "stats": stats,
@@ -437,7 +638,7 @@ def fetch_ai_news() -> dict[str, Any]:
"""按类别抓取国际 AI 时讯,返回 {enabled, hours, categories, flat, stats}。""" """按类别抓取国际 AI 时讯,返回 {enabled, hours, categories, flat, stats}。"""
if not _enabled(): if not _enabled():
return {"enabled": False, "categories": [], "flat": [], "stats": {}} return {"enabled": False, "categories": [], "flat": [], "stats": {}}
return _fetch_news(NEWS_CATEGORIES) return _fetch_news(NEWS_CATEGORIES, floor_today=_floor_today_enabled())
def fetch_cn_ai_news() -> dict[str, Any]: def fetch_cn_ai_news() -> dict[str, Any]:
@@ -480,20 +681,15 @@ def _format_news_section(
categories = news.get("categories") or [] categories = news.get("categories") or []
hours = news.get("hours", 72) hours = news.get("hours", 72)
floor_note = " · 仅今日" if news.get("floor_today") else ""
lines = [ lines = [
"---", "---",
"", "",
f"## {section_no}{title}", f"## {section_no}{title}",
"", "",
f"> 近 **{hours}h** · {news.get('stats', {}).get('feeds_ok', 0)}/{news.get('stats', {}).get('feeds_total', 0)} 源可用", f"> 近 **{hours}h**{floor_note} · {news.get('stats', {}).get('feeds_ok', 0)}/{news.get('stats', {}).get('feeds_total', 0)} 源可用",
"", "",
] ]
failed = news.get("stats", {}).get("feeds_failed") or []
if failed:
preview = "".join(failed[:5])
suffix = "" if len(failed) > 5 else ""
lines.append(f"> ⚠️ {len(failed)} 个源抓取失败:{preview}{suffix}")
lines.append("")
if not categories: if not categories:
lines.append("*暂无可用条目(网络/RSS 源异常或时间窗口内无更新)。*") lines.append("*暂无可用条目(网络/RSS 源异常或时间窗口内无更新)。*")
@@ -524,93 +720,27 @@ def _format_news_section(
return lines return lines
def _sorted_flat(news: dict[str, Any]) -> list[dict[str, Any]]: def prepare_wecom_news_items(news: dict[str, Any], *, date_str: str | None = None) -> list[dict[str, Any]]:
flat = list(news.get("flat") or [])
flat.sort(
key=lambda item: (float(item.get("score") or 0), _sort_key(item)[1]),
reverse=True,
)
return flat
def prepare_wecom_news_items(news: dict[str, Any], *, limit: int | None = None) -> list[dict[str, Any]]:
if not news.get("enabled"): if not news.get("enabled"):
return [] return []
pick_limit = limit or _wecom_limit() limit = _wecom_limit()
flat = _sorted_flat(news) floor = bool(news.get("floor_today", _floor_today_enabled()))
flat_strict = _filter_flat_in_window(news.get("flat") or [], floor_today=floor)
flat_relaxed = _filter_flat_in_window(news.get("flat") or [], floor_today=False)
preferred = ("media", "newsletter", "official", "community", "research", "developer") preferred = ("media", "newsletter", "official", "community", "research", "developer")
picked: list[dict[str, Any]] = [] picked = _pick_news_items(flat_strict, limit, preferred)
seen: set[str] = set() picked = _fill_picked_to_limit(picked, [flat_relaxed, news.get("flat") or []], limit)
for cat in preferred: items = [_to_wecom_news_row(item) for item in picked[:limit]]
for item in flat: return _apply_pushed_dedup_with_backfill(items, picked, date_str=date_str, limit=limit)
link = _normalize_link(item.get("link", ""))
if item.get("category_id") != cat or link in seen:
continue
picked.append(item)
seen.add(link)
if len(picked) >= pick_limit:
break
if len(picked) >= pick_limit:
break
items: list[dict[str, Any]] = []
for item in picked[:pick_limit]:
items.append(
{
"title": item.get("title", "?"),
"link": item.get("link", ""),
"source_name": item.get("source_name", "?"),
"published_fmt": item.get("published_fmt", ""),
"desc_short": _clean_text(item.get("summary", ""), 36),
"score": item.get("score"),
}
)
return items
def prepare_wecom_cn_news_items(news: dict[str, Any], *, limit: int | None = None) -> list[dict[str, Any]]: def prepare_wecom_cn_news_items(news: dict[str, Any], *, date_str: str | None = None) -> list[dict[str, Any]]:
if not news.get("enabled"): if not news.get("enabled"):
return [] return []
pick_limit = limit or _wecom_cn_limit() limit = _wecom_cn_limit()
flat = _sorted_flat(news) flat = _filter_flat_in_window(news.get("flat") or [], floor_today=False)
preferred = ("media", "tech", "dev") preferred = ("media", "tech")
picked: list[dict[str, Any]] = [] picked = _pick_news_items(flat, limit, preferred, one_per_source=True)
seen_links: set[str] = set() picked = _fill_picked_to_limit(picked, [news.get("flat") or []], limit)
seen_sources: set[str] = set() items = [_to_wecom_news_row(item) for item in picked[:limit]]
return _apply_pushed_dedup_with_backfill(items, picked, date_str=date_str, limit=limit)
for cat in preferred:
for item in flat:
link = _normalize_link(item.get("link", ""))
source = item.get("source_name", "?")
if item.get("category_id") != cat or not link or link in seen_links or source in seen_sources:
continue
picked.append(item)
seen_links.add(link)
seen_sources.add(source)
if len(picked) >= pick_limit:
break
if len(picked) >= pick_limit:
break
if len(picked) < pick_limit:
for item in flat:
link = _normalize_link(item.get("link", ""))
if not link or link in seen_links:
continue
picked.append(item)
seen_links.add(link)
if len(picked) >= pick_limit:
break
items: list[dict[str, Any]] = []
for item in picked[:pick_limit]:
items.append(
{
"title": item.get("title", "?"),
"link": item.get("link", ""),
"source_name": item.get("source_name", "?"),
"published_fmt": item.get("published_fmt", ""),
"desc_short": _clean_text(item.get("summary", ""), 36),
"score": item.get("score"),
}
)
return items

View File

@@ -0,0 +1,87 @@
"""已推送企微早报的新闻 link 去重缓存。"""
from __future__ import annotations
import json
from datetime import datetime, timedelta
from pathlib import Path
from typing import Any
from daily.config import CACHE_DIR, news_dedup_days
from daily.news.fetch import _normalize_link
def _cache_path() -> Path:
return CACHE_DIR / "pushed-news-links.json"
def _load_raw() -> dict[str, Any]:
path = _cache_path()
if not path.exists():
return {"dates": {}}
try:
data = json.loads(path.read_text(encoding="utf-8"))
except (OSError, ValueError):
return {"dates": {}}
if not isinstance(data.get("dates"), dict):
return {"dates": {}}
return data
def _save_raw(data: dict[str, Any]) -> None:
path = _cache_path()
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8")
def _prune(data: dict[str, Any], *, keep_days: int) -> None:
dates: dict[str, list[str]] = data.setdefault("dates", {})
try:
anchor = max(datetime.strptime(d, "%Y-%m-%d") for d in dates)
except ValueError:
return
cutoff = anchor - timedelta(days=keep_days)
for key in list(dates.keys()):
try:
if datetime.strptime(key, "%Y-%m-%d") < cutoff:
dates.pop(key, None)
except ValueError:
dates.pop(key, None)
def load_pushed_link_set() -> set[str]:
data = _load_raw()
out: set[str] = set()
for links in (data.get("dates") or {}).values():
if isinstance(links, list):
out.update(str(x) for x in links if x)
return out
def filter_unpushed_items(
items: list[dict[str, Any]],
*,
date_str: str,
) -> list[dict[str, Any]]:
del date_str # reserved for per-day scoping if needed later
seen = load_pushed_link_set()
out: list[dict[str, Any]] = []
for item in items:
link = _normalize_link(str(item.get("link") or ""))
if not link or link in seen:
continue
out.append(item)
return out
def record_pushed_links(date_str: str, links: list[str]) -> None:
data = _load_raw()
dates: dict[str, list[str]] = data.setdefault("dates", {})
normalized: list[str] = []
for link in links:
clean = _normalize_link(link)
if clean:
normalized.append(clean)
dates[date_str] = sorted(set(normalized))
_prune(data, keep_days=news_dedup_days())
_save_raw(data)

View File

@@ -1,171 +0,0 @@
"""RSS 新闻去重、打分与排序。"""
from __future__ import annotations
import logging
import re
from datetime import datetime, timezone
from typing import Any
from daily.config import env_int
from daily.delta import find_previous_data
from daily.news.fetch import _entry_datetime, _normalize_link, _normalize_title
logger = logging.getLogger(__name__)
_WS = re.compile(r"\s+")
_TOKEN = re.compile(r"[\w]{2,}", re.UNICODE)
_CJK = re.compile(r"[\u4e00-\u9fff]")
CATEGORY_TIER: dict[str, int] = {
"official": 20,
"developer": 16,
"research": 14,
"media": 12,
"newsletter": 10,
"community": 8,
"tech": 12,
"dev": 10,
}
def title_similarity_threshold() -> float:
raw = env_int("DAILY_NEWS_TITLE_SIM", 55)
return max(0, min(raw, 95)) / 100.0
def _title_tokens(title: str) -> set[str]:
normalized = _normalize_title(title)
tokens = set(_TOKEN.findall(normalized))
cjk = "".join(_CJK.findall(title))
for i in range(max(0, len(cjk) - 1)):
tokens.add(cjk[i : i + 2])
if not tokens and normalized:
tokens.add(normalized)
return tokens
def jaccard_similarity(left: set[str], right: set[str]) -> float:
if not left or not right:
return 0.0
inter = len(left & right)
union = len(left | right)
return inter / union if union else 0.0
def fuzzy_dedupe_by_title(
items: list[dict[str, Any]],
*,
threshold: float | None = None,
) -> list[dict[str, Any]]:
"""按标题相似度合并重复报道,保留 score 更高(或更靠前)的条目。"""
if not items:
return []
limit = threshold if threshold is not None else title_similarity_threshold()
kept: list[dict[str, Any]] = []
kept_tokens: list[set[str]] = []
for item in items:
tokens = _title_tokens(str(item.get("title") or ""))
duplicate_idx: int | None = None
for idx, existing_tokens in enumerate(kept_tokens):
if jaccard_similarity(tokens, existing_tokens) >= limit:
duplicate_idx = idx
break
if duplicate_idx is None:
kept.append(item)
kept_tokens.append(tokens)
continue
existing = kept[duplicate_idx]
if float(item.get("score") or 0) > float(existing.get("score") or 0):
kept[duplicate_idx] = item
kept_tokens[duplicate_idx] = tokens
return kept
def _freshness_points(item: dict[str, Any], *, now: datetime | None = None) -> int:
now = now or datetime.now(timezone.utc)
dt = _entry_datetime(item)
if dt is None:
return 4
age_hours = max(0.0, (now - dt).total_seconds() / 3600.0)
if age_hours <= 6:
return 30
if age_hours <= 24:
return 22
if age_hours <= 72:
return 8
return 0
def _source_tier(item: dict[str, Any]) -> int:
category_id = str(item.get("category_id") or "")
return CATEGORY_TIER.get(category_id, 8)
def _novelty_points(item: dict[str, Any], yesterday_links: set[str]) -> int:
link = _normalize_link(str(item.get("link") or ""))
if link and link in yesterday_links:
return -20
return 5
def score_news_item(
item: dict[str, Any],
*,
yesterday_links: set[str] | None = None,
now: datetime | None = None,
) -> int:
links = yesterday_links or set()
total = _source_tier(item) + _freshness_points(item, now=now) + _novelty_points(item, links)
return max(0, total)
def load_yesterday_news_links(date_str: str) -> set[str]:
baseline = find_previous_data(date_str)
if not baseline:
return set()
_, data = baseline
links: set[str] = set()
for key in ("ai_news", "cn_ai_news"):
for item in data.get(key) or []:
if not isinstance(item, dict):
continue
link = _normalize_link(str(item.get("link") or ""))
if link:
links.add(link)
return links
def apply_news_ranking(news: dict[str, Any], *, date_str: str) -> dict[str, Any]:
"""对 categories / flat 打分、模糊去重并写回 score 字段。"""
if not news.get("enabled"):
return news
yesterday_links = load_yesterday_news_links(date_str)
threshold = title_similarity_threshold()
now = datetime.now(timezone.utc)
categories = news.get("categories") or []
for category in categories:
items = list(category.get("items") or [])
for item in items:
item["score"] = score_news_item(item, yesterday_links=yesterday_links, now=now)
items.sort(key=lambda x: float(x.get("score") or 0), reverse=True)
category["items"] = fuzzy_dedupe_by_title(items, threshold=threshold)
flat: list[dict[str, Any]] = []
for category in categories:
flat.extend(category.get("items") or [])
flat.sort(key=lambda x: float(x.get("score") or 0), reverse=True)
news["flat"] = fuzzy_dedupe_by_title(flat, threshold=threshold)
stats = dict(news.get("stats") or {})
stats["ranked_items"] = len(news["flat"])
stats["yesterday_links"] = len(yesterday_links)
news["stats"] = stats
logger.info(
"新闻排序完成:%d 条 flat昨日链接基准 %d",
len(news["flat"]),
len(yesterday_links),
)
return news

324
daily/news/research.py Normal file
View File

@@ -0,0 +1,324 @@
"""Cursor SDK + deep-research 工作流:采集 AI 时讯(方案 A内置 WebSearch"""
from __future__ import annotations
import json
import logging
from datetime import datetime, timezone, timedelta
from pathlib import Path
from typing import Any
from urllib.parse import urlparse
from daily.config import OUTPUT_DIR, ROOT, env, env_int, wecom_ai_news_tech_limit
from daily.llm_client import cursor_agent_prompt, extract_json_object, has_cursor_configured
from daily.news.fetch import brief_news_summary, _normalize_link
from daily.news.pushed_links import filter_unpushed_items
logger = logging.getLogger(__name__)
_SKILL_DIR = ROOT / "skills" / "daily-ai-news-research"
_DEEP_RESEARCH_CANDIDATES = (
ROOT / "skills" / "deep-research" / "SKILL.md",
Path.home() / ".agents" / "skills" / "deep-research" / "SKILL.md",
Path.home() / ".cursor" / "skills" / "deep-research" / "SKILL.md",
)
def ai_news_mode() -> str:
return (env("DAILY_AI_NEWS_MODE") or "rss").strip().lower()
def is_research_mode() -> bool:
return ai_news_mode() == "research"
def research_hours() -> int:
return max(1, env_int("DAILY_AI_NEWS_HOURS", 24))
def research_limit() -> int:
return max(1, env_int("DAILY_WECOM_AI_NEWS", 10))
def research_json_path(date_str: str) -> Path:
return OUTPUT_DIR / f"{date_str}.ai-news-research.json"
def _save_research_json(path: Path, data: dict[str, Any]) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8")
def _load_skill() -> str:
parts: list[str] = []
for path in _DEEP_RESEARCH_CANDIDATES:
if path.exists():
parts.append(path.read_text(encoding="utf-8").strip())
break
local = _SKILL_DIR / "SKILL.md"
if local.exists():
parts.append(local.read_text(encoding="utf-8").strip())
if not parts:
return "你是 AI 时讯调研员,只输出 JSON。"
return "\n\n---\n\n".join(parts)
def _guess_source_name(link: str, explicit: str) -> str:
name = (explicit or "").strip()
if name:
return name
host = urlparse(link).netloc.lower().removeprefix("www.")
mapping = {
"techcrunch.com": "TechCrunch",
"theverge.com": "The Verge",
"openai.com": "OpenAI",
"anthropic.com": "Anthropic",
"arxiv.org": "arXiv",
"qbitai.com": "量子位",
"36kr.com": "36氪",
"leiphone.com": "雷锋网",
}
for key, label in mapping.items():
if host.endswith(key) or key in host:
return label
return host.split(".")[0].capitalize() if host else "?"
def _normalize_research_item(raw: dict[str, Any]) -> dict[str, Any] | None:
title = str(raw.get("title") or "").strip()
link = _normalize_link(str(raw.get("link") or ""))
if not title or not link or not link.startswith("http"):
return None
desc = brief_news_summary(str(raw.get("desc_short") or raw.get("summary") or ""))
return {
"title": title,
"link": link,
"source_name": _guess_source_name(link, str(raw.get("source_name") or "")),
"published_fmt": str(raw.get("published_fmt") or "").strip(),
"desc_short": desc,
"summary_plain": desc,
}
def research_tech_limit() -> int:
return wecom_ai_news_tech_limit()
def _parse_items_array(
items_raw: Any,
*,
limit: int,
seen: set[str],
) -> list[dict[str, Any]]:
if not isinstance(items_raw, list):
return []
out: list[dict[str, Any]] = []
for row in items_raw:
if not isinstance(row, dict):
continue
item = _normalize_research_item(row)
if not item:
continue
if item["link"] in seen:
continue
seen.add(item["link"])
out.append(item)
if len(out) >= limit:
break
return out
def parse_research_response(
raw: str,
*,
limit: int,
tech_limit: int = 0,
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
parsed = extract_json_object(raw)
seen: set[str] = set()
items = _parse_items_array(parsed.get("items"), limit=limit, seen=seen)
tech_items = _parse_items_array(parsed.get("tech_items"), limit=tech_limit, seen=seen) if tech_limit else []
return items, tech_items
def _apply_pushed_dedup(items: list[dict[str, Any]], *, date_str: str, limit: int) -> list[dict[str, Any]]:
from daily.config import news_backfill_enabled
from daily.news.sanitize import strip_relax_window_prefix
for item in items:
if item.get("desc_short"):
item["desc_short"] = strip_relax_window_prefix(str(item.get("desc_short") or ""))
fresh = filter_unpushed_items(items, date_str=date_str)
if len(fresh) >= limit:
return fresh[:limit]
if not news_backfill_enabled():
if len(fresh) < limit:
logger.info("news_short:%s", len(fresh))
return fresh[:limit]
seen = {i.get("link") for i in fresh}
for item in items:
if len(fresh) >= limit:
break
if item.get("link") not in seen:
fresh.append(item)
seen.add(item.get("link"))
return fresh[:limit]
def fetch_ai_news_research(
*,
date_str: str,
hours: int | None = None,
limit: int | None = None,
) -> dict[str, Any]:
"""Cursor Agent 调研 AI 时讯;返回 {enabled, mode, hours, items, flat, stats}。"""
h = hours if hours is not None else research_hours()
lim = limit if limit is not None else research_limit()
tech_lim = research_tech_limit()
if not has_cursor_configured():
logger.warning("DAILY_AI_NEWS_MODE=research 但未配置 CURSOR_API_KEY")
return {
"enabled": False,
"mode": "research",
"items": [],
"tech_items": [],
"flat": [],
"stats": {"error": "no_cursor_key"},
}
skill = _load_skill()
now_cst = datetime.now(timezone(timedelta(hours=8)))
tech_clause = ""
if tech_lim:
tech_clause = (
f"\n另输出 **tech_items 恰好 {tech_lim} 条**,聚焦工程技术:"
"模型/框架发布、开源项目、芯片算力、开发者工具、推理与工程实践。"
"与 items 不得重复 link。"
)
system = (
f"{skill}\n\n"
"当前执行 **早报 AI 时讯调研**。\n"
f"时间窗口:近 **{h}** 小时(截至 {now_cst.strftime('%Y-%m-%d %H:%M')} UTC+8\n"
f"输出 **恰好 {lim} 条** items按重要性排序。{tech_clause}\n"
"使用 WebSearch 检索;不要读取本项目文档或 RSS 配置。"
)
user = (
f"/deep-research 获取近 {h} 小时的 AI 人工智能新闻资讯,"
"不区分国内国外,合并精选。"
f"只输出 JSONitems 长度={lim}"
+ (f"tech_items 长度={tech_lim}" if tech_lim else "")
+ ""
)
try:
raw = cursor_agent_prompt(system, user)
except Exception as exc:
logger.warning("AI 时讯 research 失败:%s", exc)
return {
"enabled": False,
"mode": "research",
"items": [],
"tech_items": [],
"flat": [],
"stats": {"error": str(exc)},
}
if not raw:
return {
"enabled": False,
"mode": "research",
"items": [],
"tech_items": [],
"flat": [],
"stats": {"error": "empty_response"},
}
items, tech_items = parse_research_response(raw, limit=lim, tech_limit=tech_lim)
payload = extract_json_object(raw)
if payload:
_save_research_json(research_json_path(date_str), payload)
if not items and not tech_items:
logger.warning("AI 时讯 research JSON 无效或无条目")
return {
"enabled": False,
"mode": "research",
"items": [],
"tech_items": [],
"flat": [],
"stats": {"error": "invalid_json"},
}
items = _apply_pushed_dedup(items, date_str=date_str, limit=lim)
if tech_items:
tech_items = _apply_pushed_dedup(tech_items, date_str=date_str, limit=tech_lim)
logger.info("AI 时讯 research 完成:%d 条 + %d 技术", len(items), len(tech_items))
flat = [
{
"title": i["title"],
"link": i["link"],
"summary": i.get("summary_plain") or i.get("desc_short") or "",
"source_name": i["source_name"],
"published_fmt": i.get("published_fmt") or "",
"category_id": "research",
"category_name": "Deep Research",
"category_icon": "🔍",
}
for i in items + tech_items
]
return {
"enabled": True,
"mode": "research",
"hours": h,
"items": items,
"tech_items": tech_items,
"flat": flat,
"stats": {"source": "cursor_research", "items": len(items), "tech_items": len(tech_items)},
}
def format_research_news_section(
research: dict[str, Any],
*,
section_no: int,
wecom_limit: int | None = None,
) -> list[str]:
if not research.get("enabled"):
hint = research.get("stats", {}).get("error", "调研失败或未配置 CURSOR_API_KEY")
return [
"---",
"",
f"## {section_no}、AI 时讯精选Deep Research",
"",
f"*不可用:{hint}*",
"",
]
hours = research.get("hours", 24)
items = (research.get("flat") or [])[: wecom_limit or research_limit()]
lines = [
"---",
"",
f"## {section_no}、AI 时讯精选Deep Research",
"",
f"> 近 **{hours}h** · Cursor Agent WebSearch · {len(items)}",
"",
]
if not items:
lines.append("*暂无可用条目。*")
lines.append("")
return lines
for i, item in enumerate(items, 1):
pub = f" · {item['published_fmt']}" if item.get("published_fmt") else ""
lines.append(
f"{i}. **[{item['title']}]({item['link']})** · `{item['source_name']}`{pub}"
)
summary = item.get("summary") or ""
if summary:
lines.append(f" - {summary}")
lines.append("")
return lines

18
daily/news/sanitize.py Normal file
View File

@@ -0,0 +1,18 @@
"""新闻文案清洗:剥离「放宽窗口」类凑数前缀。"""
from __future__ import annotations
import re
_RELAX_PREFIX = re.compile(
r"^(?:放宽窗口|放宽至[^:]*)\s*[:]\s*",
re.UNICODE,
)
def strip_relax_window_prefix(text: str) -> str:
"""去掉开头的「放宽窗口:」/「放宽至…:」前缀。"""
raw = (text or "").strip()
if not raw:
return ""
return _RELAX_PREFIX.sub("", raw, count=1).strip()

View File

@@ -1 +0,0 @@
"""早报生成流水线子模块。"""

View File

@@ -1,145 +0,0 @@
"""抓取与结构化输入组装。"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime
from typing import Any
from daily.agent_workflow import is_agent_mode
from daily.config import env_int
from daily.delta import compare_depth
from daily.github.search import fetch_emerging_repos, fetch_topic_hot_repos
from daily.github.trending import fetch_github_trending
from daily.news.fetch import fetch_ai_news, fetch_cn_ai_news
from daily.news.rank import apply_news_ranking
from daily.report_data import build_llm_input
from daily.skills_board import load_boards
from shared.skills_data import load_feed
@dataclass
class ReportLimits:
trending_n: int
hot_n: int
skill_pool: int
wecom_trending: int
wecom_hot: int
github_limit: int
wecom_github: int
emerging_limit: int
wecom_emerging: int
topic_limit: int
wecom_topic: int
@dataclass
class ReportContext:
feed: dict[str, Any]
date_str: str
time_str: str
updated: str
trending: list[dict[str, Any]]
hot: list[dict[str, Any]]
github_trending: list[dict[str, Any]]
github_emerging: list[dict[str, Any]]
github_topic: list[dict[str, Any]]
topic_name: str
ai_news: dict[str, Any]
cn_ai_news: dict[str, Any]
limits: ReportLimits
wecom_limits: dict[str, int]
llm_input: dict[str, Any]
def resolve_limits() -> ReportLimits:
compare_n = compare_depth()
trending_n = env_int("DAILY_TRENDING_LIMIT", 150)
hot_n = max(env_int("DAILY_HOT_LIMIT", 150), compare_n)
skill_pool = max(10, env_int("DAILY_WECOM_SKILL_POOL", 200))
wecom_trending = env_int("DAILY_WECOM_TRENDING", 10)
wecom_hot = env_int("DAILY_WECOM_HOT", 10)
github_limit = env_int("DAILY_GITHUB_TRENDING_LIMIT", 10)
wecom_github = env_int("DAILY_WECOM_GITHUB_TRENDING", env_int("DAILY_WECOM_REPOS", 10))
emerging_limit = env_int("DAILY_GITHUB_EMERGING_LIMIT", 10)
wecom_emerging = env_int("DAILY_WECOM_GITHUB_EMERGING", 10)
topic_limit = env_int("DAILY_GITHUB_TOPIC_LIMIT", 10)
wecom_topic = env_int("DAILY_WECOM_GITHUB_TOPIC", 10)
return ReportLimits(
trending_n=trending_n,
hot_n=hot_n,
skill_pool=skill_pool,
wecom_trending=wecom_trending,
wecom_hot=wecom_hot,
github_limit=github_limit,
wecom_github=wecom_github,
emerging_limit=emerging_limit,
wecom_emerging=wecom_emerging,
topic_limit=topic_limit,
wecom_topic=wecom_topic,
)
def collect_report_context(now: datetime) -> ReportContext:
limits = resolve_limits()
compare_n = compare_depth()
github_fetch_n = max(limits.github_limit, compare_n, limits.wecom_github)
emerging_fetch_n = max(limits.emerging_limit, compare_n, limits.wecom_emerging)
topic_fetch_n = max(limits.topic_limit, compare_n, limits.wecom_topic)
feed = load_feed(force=True)
date_str = now.strftime("%Y-%m-%d")
time_str = now.strftime("%H:%M") + " (UTC+8)"
updated = (feed.get("updatedAt") or "")[:10]
trending, hot = load_boards(feed, trending_limit=limits.trending_n, hot_limit=limits.hot_n)
github_trending = fetch_github_trending(github_fetch_n)
seen_repos = {r["repo"] for r in github_trending}
github_emerging = fetch_emerging_repos(emerging_fetch_n, exclude=seen_repos)
seen_repos.update(r["repo"] for r in github_emerging)
topic_name, github_topic = fetch_topic_hot_repos(topic_fetch_n, exclude=seen_repos)
ai_news = apply_news_ranking(fetch_ai_news(), date_str=date_str)
cn_ai_news = apply_news_ranking(fetch_cn_ai_news(), date_str=date_str)
wecom_limits = {
"trending": limits.wecom_trending,
"hot": limits.wecom_hot,
"trending_pool": limits.skill_pool,
"hot_pool": limits.skill_pool,
"github": limits.wecom_github,
"emerging": limits.wecom_emerging,
"topic": limits.wecom_topic,
"ai_news": env_int("DAILY_WECOM_AI_NEWS", 10),
"cn_ai_news": env_int("DAILY_WECOM_CN_AI_NEWS", 8),
}
llm_input = build_llm_input(
date_str=date_str,
updated=updated,
trending=trending,
hot=hot,
github_trending=github_trending,
github_emerging=github_emerging,
github_topic=github_topic,
topic_name=topic_name,
ai_news=ai_news,
cn_ai_news=cn_ai_news,
wecom_limits=wecom_limits,
agent_mode=is_agent_mode(),
)
return ReportContext(
feed=feed,
date_str=date_str,
time_str=time_str,
updated=updated,
trending=trending,
hot=hot,
github_trending=github_trending,
github_emerging=github_emerging,
github_topic=github_topic,
topic_name=topic_name,
ai_news=ai_news,
cn_ai_news=cn_ai_news,
limits=limits,
wecom_limits=wecom_limits,
llm_input=llm_input,
)

View File

@@ -1,180 +0,0 @@
"""归档 / 企微格式化与摘要构建。"""
from __future__ import annotations
import re
import xml.etree.ElementTree as ET
from typing import Any
import certifi
import httpx
from daily.config import full_desc_limit, wecom_skill_desc_limit
from daily.github.auth import github_html_headers
from daily.news.fetch import prepare_wecom_cn_news_items, prepare_wecom_news_items
from shared.skills_data import format_installs
from daily.pipeline.snapshot import skill_id
def short_desc(text: str, limit: int = 72) -> str:
text = re.sub(r"\s+", " ", text or "").strip()
if limit <= 0 or len(text) <= limit:
return text
return text[: limit - 3] + "..."
def archive_desc(text: str) -> str:
return short_desc(text, full_desc_limit())
def wecom_desc(text: str, limit: int = 36) -> str:
return short_desc(text, limit)
def prepare_skill_item(item: dict[str, Any], prev_ids: set[str], rank: int) -> dict[str, Any]:
badge = ""
sid = skill_id(item)
if sid not in prev_ids and prev_ids:
badge = "🆕"
elif rank == 1:
badge = "👑"
installs_fmt = item.get("installs_fmt") or format_installs(item.get("installs", 0))
title = item.get("source", "?") if item.get("cluster") else item.get("title", "?")
desc = item.get("wecom_desc") or item.get("description") or item.get("cluster_titles") or ""
limit = wecom_skill_desc_limit()
desc_short = desc if item.get("wecom_desc") or limit <= 0 else wecom_desc(desc, limit)
return {
"title": title,
"source": item.get("source", "?"),
"installs_fmt": installs_fmt,
"link": item.get("link", ""),
"desc_short": desc_short,
"badge": badge,
"cluster": bool(item.get("cluster")),
"cluster_count": item.get("cluster_count"),
"cluster_titles": item.get("cluster_titles"),
}
def prepare_github_item(item: dict[str, Any]) -> dict[str, Any]:
return {**item, "desc_short": wecom_desc(item.get("description", ""), 40)}
def build_highlights(
trending: list[dict[str, Any]],
hot: list[dict[str, Any]],
github_trending: list[dict[str, Any]],
github_emerging: list[dict[str, Any]],
ai_news: dict[str, Any] | None = None,
cn_ai_news: dict[str, Any] | None = None,
) -> list[str]:
points: list[str] = []
if ai_news and ai_news.get("enabled"):
top_news = prepare_wecom_news_items(ai_news)
if top_news:
n0 = top_news[0]
pub = f" · {n0['published_fmt']}" if n0.get("published_fmt") else ""
points.append(
f"🌍 AI 时讯 [{n0['title']}]({n0['link']})`{n0.get('source_name', '?')}`{pub}"
)
elif ai_news.get("flat"):
n0 = ai_news["flat"][0]
pub = f" · {n0['published_fmt']}" if n0.get("published_fmt") else ""
points.append(f"🌍 AI 时讯 [{n0['title']}]({n0['link']})`{n0.get('source_name', '?')}`{pub}")
if cn_ai_news and cn_ai_news.get("enabled"):
top_cn = prepare_wecom_cn_news_items(cn_ai_news)
if top_cn:
n0 = top_cn[0]
pub = f" · {n0['published_fmt']}" if n0.get("published_fmt") else ""
points.append(
f"🇨🇳 国内 AI [{n0['title']}]({n0['link']})`{n0.get('source_name', '?')}`{pub}"
)
elif cn_ai_news.get("flat"):
n0 = cn_ai_news["flat"][0]
pub = f" · {n0['published_fmt']}" if n0.get("published_fmt") else ""
points.append(
f"🇨🇳 国内 AI [{n0['title']}]({n0['link']})`{n0.get('source_name', '?')}`{pub}"
)
if trending:
t0 = trending[0]
points.append(f"📈 Skills 榜首 **{t0.get('title')}**{format_installs(t0.get('installs', 0))}")
if github_trending:
g0 = github_trending[0]
stars = g0.get("stars_today_fmt", "")
total = g0.get("total_stars_fmt", "")
star_hint = f"+{stars} today · " if stars else (f"{total} · " if total else "")
points.append(f"🐙 GitHub Trending [{g0['repo']}]({g0['url']}){star_hint}{g0.get('language', '')}")
if github_emerging:
e0 = github_emerging[0]
points.append(f"🌱 新兴 [{e0['repo']}]({e0['url']})(⭐ {e0.get('total_stars_fmt', '?')}")
elif hot:
h0 = hot[0]
points.append(f"🔥 Skills Hot 榜首 **{h0.get('title')}**1H {format_installs(h0.get('installs', 0))}")
while len(points) < 3 and len(trending) > len(points):
item = trending[len(points)]
points.append(f"✨ **{item.get('title')}** · `{item.get('source')}`")
return points[:3]
def format_github_repo_section(repos: list[dict[str, Any]], *, show_created: bool = False) -> list[str]:
lines: list[str] = []
for i, repo in enumerate(repos, 1):
lang = repo.get("language") or ""
stars_today = repo.get("stars_today_fmt") or ""
total = repo.get("total_stars_fmt") or ""
created = repo.get("created_at") or ""
meta_parts = [lang]
if stars_today:
meta_parts.append(f"+{stars_today} today")
if total:
meta_parts.append(f"总 ⭐ {total}")
if show_created and created:
meta_parts.append(f"创建于 {created}")
lines.append(f"{i}. **[{repo['repo']}]({repo['url']})** · {' · '.join(meta_parts)}")
desc = archive_desc(repo.get("description", ""))
if desc:
lines.append(f" - {desc}")
lines.append("")
return lines
def format_skill_section(items: list[dict[str, Any]], *, hot: bool = False) -> list[str]:
lines: list[str] = []
for i, item in enumerate(items, 1):
sid = item.get("id") or f"{item.get('source', '?')}/{item.get('title', '?')}"
link = item.get("link", "")
installs = format_installs(item.get("installs", 0))
meta = f"1H {installs}" if hot else f"总安装 {installs}"
if link:
lines.append(f"{i}. **[{sid}]({link})** · {meta}")
else:
lines.append(f"{i}. **{sid}** · {meta}")
desc = archive_desc(item.get("description", ""))
if desc:
lines.append(f" - {desc}")
lines.append("")
return lines
def fetch_latest_release_title(repo: str) -> str | None:
atom_url = f"https://github.com/{repo}/releases.atom"
try:
with httpx.Client(
timeout=12.0,
verify=certifi.where(),
follow_redirects=True,
headers=github_html_headers(),
) as client:
resp = client.get(atom_url)
if resp.status_code != 200:
return None
root = ET.fromstring(resp.text)
ns = {"a": "http://www.w3.org/2005/Atom"}
entry = root.find("a:entry", ns)
if entry is None:
return None
title = entry.find("a:title", ns)
return title.text.strip() if title is not None and title.text else None
except Exception:
return None

View File

@@ -1,116 +0,0 @@
"""归档内容中文化。"""
from __future__ import annotations
from typing import Any
from daily.config import full_desc_limit, news_summary_limit
from daily.localize import LocalizeJob, localize_descriptions, needs_chinese
from daily.pipeline.snapshot import skill_id
def localize_descriptions_in_place(
trending: list[dict[str, Any]],
hot: list[dict[str, Any]],
github_trending: list[dict[str, Any]],
github_emerging: list[dict[str, Any]],
github_topic: list[dict[str, Any]],
ai_news: dict[str, Any],
cn_ai_news: dict[str, Any] | None = None,
) -> None:
full_limit = full_desc_limit()
news_limit = news_summary_limit()
jobs: list[LocalizeJob] = []
seen_skill: set[str] = set()
for item in trending + hot:
sid = skill_id(item)
if sid in seen_skill:
continue
seen_skill.add(sid)
desc = (item.get("description") or "").strip()
if desc:
jobs.append(LocalizeJob(f"skill:{sid}", desc, full_limit))
seen_repo: set[str] = set()
for repo_list in (github_trending, github_emerging, github_topic):
for item in repo_list:
repo = item.get("repo", "")
if not repo or repo in seen_repo:
continue
seen_repo.add(repo)
desc = (item.get("description") or "").strip()
if desc:
jobs.append(LocalizeJob(f"github:{repo}", desc, full_limit))
if ai_news.get("enabled"):
seen_news: set[str] = set()
for item in ai_news.get("flat") or []:
link = item.get("link", "")
if not link or link in seen_news:
continue
seen_news.add(link)
summary = (item.get("summary") or "").strip()
if summary:
jobs.append(LocalizeJob(f"news:{link}", summary, news_limit))
zh_map = localize_descriptions(jobs, archive=True)
if not zh_map and not jobs:
return
def _apply_zh(mapping: dict[str, str]) -> None:
for item in trending + hot:
key = f"skill:{skill_id(item)}"
if key in mapping:
item["description"] = mapping[key]
for repo_list in (github_trending, github_emerging, github_topic):
for item in repo_list:
key = f"github:{item.get('repo', '')}"
if key in mapping:
item["description"] = mapping[key]
if ai_news.get("enabled"):
for cat in ai_news.get("categories") or []:
for item in cat.get("items") or []:
key = f"news:{item.get('link', '')}"
if key in mapping:
item["summary"] = mapping[key]
for item in ai_news.get("flat") or []:
key = f"news:{item.get('link', '')}"
if key in mapping:
item["summary"] = mapping[key]
_apply_zh(zh_map)
retry_jobs: list[LocalizeJob] = []
seen_skill.clear()
for item in trending + hot:
sid = skill_id(item)
if sid in seen_skill:
continue
seen_skill.add(sid)
desc = (item.get("description") or "").strip()
if needs_chinese(desc):
retry_jobs.append(LocalizeJob(f"skill:{sid}", desc, full_limit))
seen_repo.clear()
for repo_list in (github_trending, github_emerging, github_topic):
for item in repo_list:
repo = item.get("repo", "")
if not repo or repo in seen_repo:
continue
seen_repo.add(repo)
desc = (item.get("description") or "").strip()
if needs_chinese(desc):
retry_jobs.append(LocalizeJob(f"github:{repo}", desc, full_limit))
if ai_news.get("enabled"):
seen_news.clear()
for item in ai_news.get("flat") or []:
link = item.get("link", "")
if not link or link in seen_news:
continue
seen_news.add(link)
summary = (item.get("summary") or "").strip()
if needs_chinese(summary):
retry_jobs.append(LocalizeJob(f"news:{link}", summary, news_limit))
if retry_jobs:
_apply_zh(localize_descriptions(retry_jobs, archive=True))

View File

@@ -1,36 +0,0 @@
"""Skills 快照读写。"""
from __future__ import annotations
import json
from typing import Any
from daily.config import CACHE_DIR, SNAPSHOT_FILE
def skill_id(item: dict[str, Any]) -> str:
return str(item.get("id") or f"{item.get('source')}/{item.get('title')}")
def load_snapshot() -> set[str]:
if not SNAPSHOT_FILE.exists():
return set()
try:
data = json.loads(SNAPSHOT_FILE.read_text(encoding="utf-8"))
return set(str(x) for x in (data.get("skill_ids") or []))
except (OSError, json.JSONDecodeError):
return set()
def save_snapshot(feed: dict[str, Any], date_str: str) -> None:
CACHE_DIR.mkdir(parents=True, exist_ok=True)
ids: list[str] = []
for board in ("topTrending", "topHot"):
for item in feed.get(board, [])[:20]:
sid = skill_id(item)
if sid not in ids:
ids.append(sid)
SNAPSHOT_FILE.write_text(
json.dumps({"date": date_str, "skill_ids": ids}, ensure_ascii=False, indent=2),
encoding="utf-8",
)

View File

@@ -1,53 +0,0 @@
"""主题检测与聚类。"""
from __future__ import annotations
from collections import defaultdict
from typing import Any
THEME_RULES: list[tuple[str, str, list[str]]] = [
("🎬", "AI 多媒体 / 视频", ["runcomfy", "remotion", "video", "seedance", "inpaint", "lipsync"]),
("🔧", "工程协作 / Skill 元能力", ["grill", "tdd", "architecture", "find-skills", "to-issues"]),
("📱", "飞书 / Lark", ["lark", "feishu"]),
("📣", "内容营销", ["viral", "tiktok", "instagram", "reels"]),
("🎨", "设计 / 前端", ["frontend", "design", "ui-ux", "tailwind"]),
]
def detect_theme_line(feed: dict[str, Any]) -> str:
scores: dict[str, int] = defaultdict(int)
for board in ("topTrending", "topHot"):
for rank, item in enumerate(feed.get(board, [])[:10], 1):
haystack = " ".join(
[item.get("title", ""), item.get("source", ""), item.get("description", "")]
).lower()
for _icon, label, keywords in THEME_RULES:
if any(k in haystack for k in keywords):
scores[label] += max(1, 11 - rank)
break
if not scores:
return "**今日主题**Agent Skills 生态持续活跃"
return f"**今日主题**{max(scores.items(), key=lambda x: x[1])[0]}"
def theme_clusters(feed: dict[str, Any], limit: int = 5) -> list[tuple[str, list[str]]]:
from daily.pipeline.snapshot import skill_id
buckets: dict[str, list[str]] = defaultdict(list)
seen: set[str] = set()
for board in ("topTrending", "topHot"):
for item in feed.get(board, [])[:20]:
item_id = skill_id(item)
if item_id in seen:
continue
seen.add(item_id)
haystack = " ".join(
[item.get("title", ""), item.get("source", ""), item.get("description", "")]
).lower()
for _icon, theme, keywords in THEME_RULES:
if any(k in haystack for k in keywords):
label = f"**{item.get('title')}** (`{item.get('source')}`)"
if label not in buckets[theme]:
buckets[theme].append(label)
break
return [(theme, examples[:limit]) for theme, examples in buckets.items() if examples]

61
daily/push_gate.py Normal file
View File

@@ -0,0 +1,61 @@
"""企微早报推送闸门。"""
from __future__ import annotations
import logging
from dataclasses import dataclass, field
from typing import Any
from daily.config import force_push, skip_push_when_silent
logger = logging.getLogger(__name__)
@dataclass
class PushGateResult:
should_push: bool
reasons: list[str] = field(default_factory=list)
silent: bool = False
def _has_board_moves(movement: dict[str, Any]) -> bool:
keys = (
"skills_trending_moves",
"skills_hot_moves",
"github_trending_moves",
"github_emerging_moves",
"github_topic_moves",
)
return any(movement.get(k) for k in keys)
def evaluate_push_gate(
*,
movement: dict[str, Any],
ai_news_items: list[dict[str, Any]],
cn_ai_news_items: list[dict[str, Any]],
featured_pick: dict[str, Any] | None,
) -> PushGateResult:
if force_push():
logger.info("push_gate: force_push=on, 强制推送")
return PushGateResult(should_push=True, reasons=["force_push"], silent=False)
reasons: list[str] = []
if _has_board_moves(movement):
reasons.append("board_moves")
if ai_news_items:
reasons.append("ai_news")
if cn_ai_news_items:
reasons.append("cn_ai_news")
if featured_pick:
reasons.append("featured_pick")
should = bool(reasons)
silent = not should and skip_push_when_silent()
if should:
logger.info("push_gate: 推送 (原因: %s)", ",".join(reasons))
elif silent:
logger.info("push_gate: 静默日, 跳过推送 (无任何更新信号)")
else:
logger.info("push_gate: 无更新但 skip_push_when_silent=off, 仍推送")
return PushGateResult(should_push=should, reasons=reasons, silent=silent)

View File

@@ -6,10 +6,11 @@ import json
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
from daily.config import OUTPUT_DIR, env_int from daily.config import OUTPUT_DIR, env_int, wecom_mode
from daily.delta import build_movement_baseline, build_movement_context, compare_depth from daily.delta import build_movement_baseline, build_movement_context, compare_depth, effective_wecom_mode
from daily.news.fetch import prepare_wecom_cn_news_items, prepare_wecom_news_items from daily.news.fetch import prepare_wecom_cn_news_items, prepare_wecom_news_items
from daily.skills_group import group_skills_by_source from daily.skills_group import group_skills_by_source
from daily.text_utils import trim_brief
def skill_id(item: dict[str, Any]) -> str: def skill_id(item: dict[str, Any]) -> str:
@@ -59,51 +60,41 @@ def _slim_news_items(
limit: int, limit: int,
*, *,
prepare=prepare_wecom_news_items, prepare=prepare_wecom_news_items,
date_str: str | None = None,
) -> list[dict[str, Any]]: ) -> list[dict[str, Any]]:
items: list[dict[str, Any]] = [] items: list[dict[str, Any]] = []
for item in prepare(ai_news, limit=limit): for item in prepare(ai_news, date_str=date_str):
payload = { items.append(
"link": item.get("link", ""), {
"title": item.get("title", ""), "link": item.get("link", ""),
"source_name": item.get("source_name", ""), "title": item.get("title", ""),
"published_fmt": item.get("published_fmt", ""), "source_name": item.get("source_name", ""),
"summary": item.get("desc_short") or "", "published_fmt": item.get("published_fmt", ""),
} "summary": trim_brief(
if item.get("score") is not None: item.get("summary_plain") or item.get("desc_short") or "",
payload["score"] = item.get("score") 120,
items.append(payload) ),
}
)
if len(items) >= limit: if len(items) >= limit:
break break
if items: if items:
return items return items
flat = sorted( for item in (ai_news.get("flat") or [])[:limit]:
ai_news.get("flat") or [], items.append(
key=lambda row: float(row.get("score") or 0), {
reverse=True, "link": item.get("link", ""),
) "title": item.get("title", ""),
for item in flat[:limit]: "source_name": item.get("source_name", ""),
payload = { "published_fmt": item.get("published_fmt", ""),
"link": item.get("link", ""), "summary": item.get("summary", ""),
"title": item.get("title", ""), }
"source_name": item.get("source_name", ""), )
"published_fmt": item.get("published_fmt", ""),
"summary": item.get("summary", ""),
}
if item.get("score") is not None:
payload["score"] = item.get("score")
items.append(payload)
return items return items
def _news_pool_limit(wecom_limit: int, *, env_key: str, default_pool: int, agent_mode: bool) -> int:
if not agent_mode:
return wecom_limit
pool = env_int(env_key, default_pool)
return max(wecom_limit, pool)
def _wecom_skill_pool() -> int: def _wecom_skill_pool() -> int:
return max(10, env_int("DAILY_WECOM_SKILL_POOL", 200)) return max(10, env_int("DAILY_WECOM_SKILL_POOL", 400))
def build_llm_input( def build_llm_input(
@@ -119,21 +110,46 @@ def build_llm_input(
ai_news: dict[str, Any], ai_news: dict[str, Any],
cn_ai_news: dict[str, Any], cn_ai_news: dict[str, Any],
wecom_limits: dict[str, int], wecom_limits: dict[str, int],
agent_mode: bool = False, research_items: list[dict[str, Any]] | None = None,
research_tech_items: list[dict[str, Any]] | None = None,
boards_for_wecom: dict[str, list[dict[str, Any]]] | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
"""供 Cursor 编辑的精简 JSON不含完整 markdown""" """供 Cursor 编辑的精简 JSON不含完整 markdown"""
news_limit = _news_pool_limit( news_limit = wecom_limits.get("ai_news", 10)
wecom_limits.get("ai_news", 10), cn_news_limit = wecom_limits.get("cn_ai_news", 8)
env_key="DAILY_AGENT_NEWS_POOL", if research_items is not None:
default_pool=40, slim_research = [
agent_mode=agent_mode, {
) "link": item.get("link", ""),
cn_news_limit = _news_pool_limit( "title": item.get("title", ""),
wecom_limits.get("cn_ai_news", 8), "source_name": item.get("source_name", ""),
env_key="DAILY_AGENT_CN_NEWS_POOL", "published_fmt": item.get("published_fmt", ""),
default_pool=30, "summary": trim_brief(item.get("desc_short") or "", 120),
agent_mode=agent_mode, }
) for item in research_items[:news_limit]
]
ai_news_payload = slim_research
tech_news_payload = [
{
"link": item.get("link", ""),
"title": item.get("title", ""),
"source_name": item.get("source_name", ""),
"published_fmt": item.get("published_fmt", ""),
"summary": trim_brief(item.get("desc_short") or "", 120),
}
for item in (research_tech_items or [])
]
cn_news_payload: list[dict[str, Any]] = []
ai_news_mode = "research"
else:
ai_news_payload = _slim_news_items(ai_news, news_limit, date_str=date_str) if ai_news.get("enabled") else []
cn_news_payload = (
_slim_news_items(cn_ai_news, cn_news_limit, prepare=prepare_wecom_cn_news_items, date_str=date_str)
if cn_ai_news.get("enabled")
else []
)
ai_news_mode = "rss"
tech_news_payload: list[dict[str, Any]] = []
depth = compare_depth() depth = compare_depth()
trend_cmp = trending[:depth] trend_cmp = trending[:depth]
hot_cmp = hot[:depth] hot_cmp = hot[:depth]
@@ -141,19 +157,26 @@ def build_llm_input(
emerging_cmp = github_emerging[:depth] emerging_cmp = github_emerging[:depth]
topic_cmp = github_topic[:depth] topic_cmp = github_topic[:depth]
trending_slice = group_skills_by_source( if boards_for_wecom:
trending, trending_slice = boards_for_wecom.get("skills_trending") or []
limit=wecom_limits.get("trending", 10), hot_slice = boards_for_wecom.get("skills_hot") or []
pool_size=wecom_limits.get("trending_pool", _wecom_skill_pool()), github_slice = boards_for_wecom.get("github_trending") or []
) emerging_slice = boards_for_wecom.get("github_emerging") or []
hot_slice = group_skills_by_source( topic_slice = boards_for_wecom.get("github_topic") or []
hot, else:
limit=wecom_limits.get("hot", 10), trending_slice = group_skills_by_source(
pool_size=wecom_limits.get("hot_pool", _wecom_skill_pool()), trending,
) limit=wecom_limits.get("trending", 10),
github_slice = github_trending[: wecom_limits.get("github", 5)] pool_size=wecom_limits.get("trending_pool", _wecom_skill_pool()),
emerging_slice = github_emerging[: wecom_limits.get("emerging", 3)] )
topic_slice = github_topic[: wecom_limits.get("topic", 3)] hot_slice = group_skills_by_source(
hot,
limit=wecom_limits.get("hot", 10),
pool_size=wecom_limits.get("hot_pool", _wecom_skill_pool()),
)
github_slice = github_trending[: wecom_limits.get("github", 5)]
emerging_slice = github_emerging[: wecom_limits.get("emerging", 3)]
topic_slice = github_topic[: wecom_limits.get("topic", 3)]
movement = build_movement_context( movement = build_movement_context(
date_str=date_str, date_str=date_str,
@@ -172,10 +195,13 @@ def build_llm_input(
github_topic=[_slim_github(x) for x in topic_cmp], github_topic=[_slim_github(x) for x in topic_cmp],
depth=depth, depth=depth,
) )
eff_mode = effective_wecom_mode(date_str=date_str)
return { return {
"date": date_str, "date": date_str,
"data_updated": updated, "data_updated": updated,
"wecom_mode": wecom_mode(),
"effective_wecom_mode": eff_mode,
"skills_trending": [_slim_skill(x) for x in trending_slice], "skills_trending": [_slim_skill(x) for x in trending_slice],
"skills_hot": [_slim_skill(x) for x in hot_slice], "skills_hot": [_slim_skill(x) for x in hot_slice],
"github_trending": [_slim_github(x) for x in github_slice], "github_trending": [_slim_github(x) for x in github_slice],
@@ -184,12 +210,10 @@ def build_llm_input(
"topic": topic_name, "topic": topic_name,
"repos": [_slim_github(x) for x in topic_slice], "repos": [_slim_github(x) for x in topic_slice],
}, },
"ai_news": _slim_news_items(ai_news, news_limit) if ai_news.get("enabled") else [], "ai_news": ai_news_payload,
"cn_ai_news": _slim_news_items( "tech_ai_news": tech_news_payload,
cn_ai_news, cn_news_limit, prepare=prepare_wecom_cn_news_items "cn_ai_news": cn_news_payload,
) "ai_news_mode": ai_news_mode,
if cn_ai_news.get("enabled")
else [],
"movement": movement, "movement": movement,
"movement_baseline": movement_baseline, "movement_baseline": movement_baseline,
} }
@@ -211,6 +235,10 @@ def editorial_json_path(date_str: str) -> Path:
return OUTPUT_DIR / f"{date_str}.editorial.json" return OUTPUT_DIR / f"{date_str}.editorial.json"
def featured_json_path(date_str: str) -> Path:
return OUTPUT_DIR / f"{date_str}.featured.json"
def save_json(path: Path, data: dict[str, Any]) -> None: def save_json(path: Path, data: dict[str, Any]) -> None:
path.parent.mkdir(parents=True, exist_ok=True) path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8") path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8")

284
daily/scheduler.py Normal file
View File

@@ -0,0 +1,284 @@
"""常驻调度:按配置时刻生成早报并推送企微。"""
from __future__ import annotations
import json
import logging
import subprocess
import sys
import time
from dataclasses import dataclass
from datetime import date, datetime, time as dt_time, timedelta
from pathlib import Path
from zoneinfo import ZoneInfo
from daily.config import (
CACHE_DIR,
LOG_DIR,
OUTPUT_DIR,
ROOT,
schedule_generate_at,
schedule_push_at,
schedule_timezone_name,
)
logger = logging.getLogger(__name__)
_STATE_FILE = CACHE_DIR / "scheduler-state.json"
_POLL_SECONDS = 15
@dataclass(frozen=True)
class ClockTime:
hour: int
minute: int
@dataclass
class SchedulerState:
last_generate_date: str | None = None
last_push_date: str | None = None
@classmethod
def load(cls) -> SchedulerState:
if not _STATE_FILE.exists():
return cls()
try:
raw = json.loads(_STATE_FILE.read_text(encoding="utf-8"))
except (OSError, ValueError):
return cls()
if not isinstance(raw, dict):
return cls()
return cls(
last_generate_date=raw.get("last_generate_date"),
last_push_date=raw.get("last_push_date"),
)
def save(self) -> None:
_STATE_FILE.parent.mkdir(parents=True, exist_ok=True)
payload = {
"last_generate_date": self.last_generate_date,
"last_push_date": self.last_push_date,
}
_STATE_FILE.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
def parse_hhmm(value: str) -> ClockTime:
raw = (value or "").strip()
parts = raw.split(":", 1)
if len(parts) != 2:
raise ValueError(f"无效时间格式: {value!r},应为 HH:MM")
hour = int(parts[0])
minute = int(parts[1])
if not (0 <= hour <= 23 and 0 <= minute <= 59):
raise ValueError(f"无效时间: {value!r}")
return ClockTime(hour=hour, minute=minute)
def load_timezone() -> ZoneInfo:
name = schedule_timezone_name()
try:
return ZoneInfo(name)
except Exception as exc:
raise RuntimeError(f"无效时区 DAILY_SCHEDULE_TZ={name!r}") from exc
def _localize(day: date, clock: ClockTime, tz: ZoneInfo) -> datetime:
return datetime.combine(day, dt_time(clock.hour, clock.minute), tz)
def next_occurrence_after(clock: ClockTime, tz: ZoneInfo, after: datetime) -> datetime:
local = after.astimezone(tz)
candidate = local.replace(hour=clock.hour, minute=clock.minute, second=0, microsecond=0)
if candidate <= local:
candidate += timedelta(days=1)
return candidate
def _today_slot(day: date, clock: ClockTime, tz: ZoneInfo) -> datetime:
return _localize(day, clock, tz)
def _run_daily_subcommand(subcmd: str, *extra: str) -> int:
cmd = [sys.executable, "-m", "daily", subcmd, *extra]
logger.info("执行: %s", " ".join(cmd))
proc = subprocess.run(cmd, cwd=str(ROOT), check=False)
return int(proc.returncode)
def run_generate() -> int:
return _run_daily_subcommand("generate")
def run_push_for_date(date_str: str) -> int:
report = OUTPUT_DIR / f"{date_str}.wecom.md"
if not report.exists():
logger.error("推送失败:报告不存在 %s", report)
return 1
return _run_daily_subcommand("push", str(report))
def _setup_logging() -> Path:
LOG_DIR.mkdir(parents=True, exist_ok=True)
log_path = LOG_DIR / "scheduler.log"
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(message)s",
handlers=[
logging.FileHandler(log_path, encoding="utf-8"),
logging.StreamHandler(sys.stdout),
],
)
return log_path
def _sleep_until(target: datetime, tz: ZoneInfo) -> None:
while True:
now = datetime.now(tz)
seconds = (target - now).total_seconds()
if seconds <= 0:
return
time.sleep(min(seconds, _POLL_SECONDS))
def plan_next_action(
*,
now: datetime,
tz: ZoneInfo,
state: SchedulerState,
generate_at: ClockTime,
push_at: ClockTime,
) -> tuple[datetime, str] | None:
"""返回下一次应执行的动作;若今日已全部完成则返回明日 generate。"""
today = now.astimezone(tz).date()
today_str = today.isoformat()
gen_done = state.last_generate_date == today_str
push_done = state.last_push_date == today_str
gen_slot = _today_slot(today, generate_at, tz)
push_slot = _today_slot(today, push_at, tz)
# 推送窗口内generate 未做则立即补跑(须先于 push
if not gen_done and gen_slot <= now <= push_slot:
return now, "generate"
# 已过推送时刻:仅当 generate 已完成时补跑 push
if not push_done and gen_done and now >= push_slot:
return now, "push"
candidates: list[tuple[datetime, str]] = []
if not gen_done and gen_slot > now:
candidates.append((gen_slot, "generate"))
if not push_done and push_slot > now:
candidates.append((push_slot, "push"))
if candidates:
return min(candidates, key=lambda item: item[0])
tomorrow_gen = next_occurrence_after(generate_at, tz, now)
return tomorrow_gen, "generate"
def run_scheduled_action(action: str, *, today_str: str) -> int:
if action == "generate":
return run_generate()
if action == "push":
return run_push_for_date(today_str)
raise ValueError(f"未知动作: {action}")
def tick_once(
*,
now: datetime | None = None,
tz: ZoneInfo | None = None,
state: SchedulerState | None = None,
generate_at: ClockTime | None = None,
push_at: ClockTime | None = None,
dry_run: bool = False,
) -> SchedulerState:
tz = tz or load_timezone()
now = now or datetime.now(tz)
state = state or SchedulerState.load()
generate_at = generate_at or parse_hhmm(schedule_generate_at())
push_at = push_at or parse_hhmm(schedule_push_at())
today_str = now.astimezone(tz).date().isoformat()
planned = plan_next_action(
now=now,
tz=tz,
state=state,
generate_at=generate_at,
push_at=push_at,
)
if not planned:
return state
run_at, action = planned
if run_at > now:
if not dry_run:
logger.info("下次 %s @ %s (%s)", action, run_at.isoformat(), tz.key)
_sleep_until(run_at, tz)
elif not dry_run:
slot = _today_slot(now.astimezone(tz).date(), generate_at if action == "generate" else push_at, tz)
logger.info(
"补跑 %s(计划 %02d:%02d,当前 %s",
action,
slot.hour,
slot.minute,
now.astimezone(tz).strftime("%H:%M"),
)
if dry_run:
logger.info("[dry-run] 将执行 %s @ %s", action, run_at.isoformat())
return state
logger.info("开始 %s%s", action, today_str)
code = run_scheduled_action(action, today_str=today_str)
if code != 0:
logger.error("%s 失败exit=%s", action, code)
else:
if action == "generate":
state.last_generate_date = today_str
elif action == "push":
state.last_push_date = today_str
state.save()
logger.info("%s 完成", action)
return state
def main() -> int:
dry_run = "--dry-run" in sys.argv[1:]
once = "--once" in sys.argv[1:]
log_path = _setup_logging()
tz = load_timezone()
generate_at = parse_hhmm(schedule_generate_at())
push_at = parse_hhmm(schedule_push_at())
logger.info(
"调度器启动 tz=%s generate=%02d:%02d push=%02d:%02d log=%s",
tz.key,
generate_at.hour,
generate_at.minute,
push_at.hour,
push_at.minute,
log_path,
)
state = SchedulerState.load()
try:
while True:
state = tick_once(
tz=tz,
state=state,
generate_at=generate_at,
push_at=push_at,
dry_run=dry_run,
)
if once or dry_run:
break
except KeyboardInterrupt:
logger.info("调度器已停止KeyboardInterrupt")
return 0
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -2,20 +2,97 @@
from __future__ import annotations from __future__ import annotations
import json
import logging import logging
import re import re
import time
from typing import Any, Literal from typing import Any, Literal
import certifi import certifi
import httpx import httpx
from daily.config import env from daily.config import CACHE_DIR, env
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
Board = Literal["trending", "hot"] Board = Literal["trending", "hot"]
SKILLS_SITE = "https://www.skills.sh" SKILLS_SITE = "https://www.skills.sh"
USER_AGENT = "Mozilla/5.0 (compatible; daily-robots/1.0; +https://skills.sh)" USER_AGENT = "Mozilla/5.0 (compatible; skills-hot-daily/1.0; +https://skills.sh)"
FEED_URLS = [
"https://cdn.jsdelivr.net/gh/NeverSight/skills.sh_feed@main/data/feed.json",
"https://raw.githubusercontent.com/NeverSight/skills.sh_feed/main/data/feed.json",
]
FEED_CACHE_TTL = 600
FEED_CACHE_FILE = CACHE_DIR / "feed.json"
_feed_cache: dict[str, Any] = {"data": None, "fetched_at": 0.0}
def format_installs(n: int | float) -> str:
if n >= 1_000_000:
return f"{n / 1_000_000:.1f}M"
if n >= 1_000:
return f"{n / 1_000:.1f}K"
return str(int(n))
def _fetch_feed_json(url: str) -> dict[str, Any]:
headers = {"User-Agent": USER_AGENT, "Accept": "application/json"}
with httpx.Client(
timeout=httpx.Timeout(20.0, connect=10.0),
verify=certifi.where(),
follow_redirects=True,
) as client:
resp = client.get(url, headers=headers)
resp.raise_for_status()
return resp.json()
def _load_feed_disk_cache() -> dict[str, Any] | None:
if not FEED_CACHE_FILE.exists():
return None
try:
return json.loads(FEED_CACHE_FILE.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
logger.warning("读取 feed 本地缓存失败: %s", exc)
return None
def _save_feed_disk_cache(data: dict[str, Any]) -> None:
FEED_CACHE_FILE.parent.mkdir(parents=True, exist_ok=True)
FEED_CACHE_FILE.write_text(json.dumps(data, ensure_ascii=False), encoding="utf-8")
def load_feed(force: bool = False) -> dict[str, Any]:
now = time.time()
if not force and _feed_cache["data"] and now - _feed_cache["fetched_at"] < FEED_CACHE_TTL:
return _feed_cache["data"]
errors: list[str] = []
for url in FEED_URLS:
for attempt in range(3):
try:
data = _fetch_feed_json(url)
_feed_cache["data"] = data
_feed_cache["fetched_at"] = now
_save_feed_disk_cache(data)
logger.info("skills feed 已更新: %s", url)
return data
except Exception as exc:
msg = f"{url} (#{attempt + 1}): {exc}"
errors.append(msg)
logger.debug("拉取失败 %s", msg)
time.sleep(0.5 * (attempt + 1))
stale = _load_feed_disk_cache()
if stale:
logger.warning("网络不可用,回退到 feed 本地缓存")
_feed_cache["data"] = stale
_feed_cache["fetched_at"] = now
return stale
raise RuntimeError(f"无法获取 skills 数据。最近错误: {errors[-1] if errors else 'unknown'}")
_SKILL_RE = re.compile( _SKILL_RE = re.compile(
r'\{"source":"(?P<source>[^"]+)","skillId":"(?P<skill_id>[^"]+)",' r'\{"source":"(?P<source>[^"]+)","skillId":"(?P<skill_id>[^"]+)",'

View File

@@ -9,6 +9,28 @@ def skill_id(item: dict[str, Any]) -> str:
return str(item.get("id") or f"{item.get('source')}/{item.get('title')}") return str(item.get("id") or f"{item.get('source')}/{item.get('title')}")
def source_from_skill_key(key: str) -> str:
"""从 skill idsource/title…还原 source去掉最后一段 title。"""
parts = [p for p in str(key or "").split("/") if p]
if len(parts) >= 2:
return "/".join(parts[:-1])
return str(key or "").strip()
def expand_skill_recent_keys(keys: set[str] | None) -> set[str]:
"""周去重 blocklist保留原始 key并展开为 source。"""
out: set[str] = set()
for key in keys or set():
k = str(key or "").strip()
if not k:
continue
out.add(k)
src = source_from_skill_key(k)
if src:
out.add(src)
return out
def format_installs(n: int | float) -> str: def format_installs(n: int | float) -> str:
if n >= 1_000_000: if n >= 1_000_000:
return f"{n / 1_000_000:.1f}M" return f"{n / 1_000_000:.1f}M"

View File

@@ -15,7 +15,7 @@ def clip_text(text: str, limit: int) -> str:
def trim_brief(text: str, limit: int) -> str: def trim_brief(text: str, limit: int) -> str:
"""企微简要:控制在 limit 内,优先在句读截断,不加省略号。""" """企微简要:控制在 limit 内,优先在句读/词边界截断,不加省略号。"""
text = _WS.sub(" ", (text or "").strip()) text = _WS.sub(" ", (text or "").strip())
if not text or limit <= 0 or len(text) <= limit: if not text or limit <= 0 or len(text) <= limit:
return text return text
@@ -27,4 +27,12 @@ def trim_brief(text: str, limit: int) -> str:
pos = text.find(sep) pos = text.find(sep)
if pos != -1 and pos + 1 <= limit: if pos != -1 and pos + 1 <= limit:
return text[: pos + 1] return text[: pos + 1]
return text[:limit].rstrip(",、;: ") for sep in (". ", "! ", "? ", "; "):
pos = text.rfind(sep, 0, limit + 1)
if pos != -1 and pos + 1 >= min(limit // 2, 20):
return text[: pos + 1].rstrip()
if len(text) > limit:
space = text.rfind(" ", 0, limit + 1)
if space >= min(limit // 2, 20):
return text[:space].rstrip(",、;: ,.;")
return text[:limit].rstrip(",、;: ,.;")

View File

@@ -2,6 +2,8 @@
from __future__ import annotations from __future__ import annotations
import json
import re
import sys import sys
import time import time
from pathlib import Path from pathlib import Path
@@ -13,6 +15,7 @@ from daily.config import OUTPUT_DIR, ROOT, env_int, wecom_chunk_bytes
from daily.wecom_split import split_wecom_messages from daily.wecom_split import split_wecom_messages
_PUSH_GAP_MS = 300 _PUSH_GAP_MS = 300
_DATE_RE = re.compile(r"(\d{4}-\d{2}-\d{2})\.wecom\.md$")
def _load_webhook_key() -> str: def _load_webhook_key() -> str:
@@ -43,6 +46,31 @@ def _resolve_report_path(arg: str | None) -> Path:
raise RuntimeError("未找到 .wecom.md 报告,请先运行 python -m daily") raise RuntimeError("未找到 .wecom.md 报告,请先运行 python -m daily")
def _push_gate_for_report(path: Path) -> dict | None:
match = _DATE_RE.search(path.name)
if not match:
return None
data_path = path.parent / f"{match.group(1)}.data.json"
if not data_path.exists():
data_path = OUTPUT_DIR / f"{match.group(1)}.data.json"
if not data_path.exists():
return None
try:
payload = json.loads(data_path.read_text(encoding="utf-8"))
except (OSError, ValueError):
return None
meta = payload.get("meta") or {}
gate = meta.get("push_gate")
return gate if isinstance(gate, dict) else None
def should_skip_push(report_path: Path) -> bool:
gate = _push_gate_for_report(report_path)
if not gate:
return False
return bool(gate.get("silent")) and not gate.get("should_push")
def _post_markdown(client: httpx.Client, url: str, content: str) -> None: def _post_markdown(client: httpx.Client, url: str, content: str) -> None:
payload = {"msgtype": "markdown", "markdown": {"content": content}} payload = {"msgtype": "markdown", "markdown": {"content": content}}
resp = client.post(url, json=payload) resp = client.post(url, json=payload)
@@ -54,6 +82,9 @@ def _post_markdown(client: httpx.Client, url: str, content: str) -> None:
def send_report(report_path: Path | None = None) -> None: def send_report(report_path: Path | None = None) -> None:
path = _resolve_report_path(str(report_path) if report_path else None) path = _resolve_report_path(str(report_path) if report_path else None)
if should_skip_push(path):
print(f"[silent] no push gate matched for {path.name}")
return
if not path.exists(): if not path.exists():
raise RuntimeError(f"报告文件不存在: {path}") raise RuntimeError(f"报告文件不存在: {path}")

View File

@@ -1,14 +0,0 @@
services:
daily:
build: .
env_file: .env
environment:
TZ: Asia/Shanghai
volumes:
- ./output:/app/output
- ./.cache:/app/.cache
- ./logs:/app/logs
# Examples:
# docker compose run --rm daily
# docker compose run --rm daily --skip-push
# docker compose run --rm daily --force

View File

@@ -0,0 +1,273 @@
# Design: 企微早报 Delta 模式
Generated: 2026-07-09
Repo: daily-robots
Status: DRAFT
Mode: Builder
## Problem Statement
企微早报每天推送五榜 Top 10 + 18 条新闻,内容与前几日高度重复(`find-skills``openclaw`、飞书集群等长期霸榜)。读者真实需求是「今天有什么新变化」,而非「再读一遍黄页」。
根因:
1. `daily-agent/SKILL.md` 要求即使较昨日无新增,仍须完整列出 Top 榜。
2. `movement` 仅用于 opening / signals列表区块仍全量渲染。
3. Trending 与 Hot 独立展示,同一 skill 描述写两遍。
4. 新闻 `DAILY_AI_NEWS_HOURS=72`,无已推送 link 去重,旧闻可连续出现。
## What Makes This Cool
把早报从「日报复印机」变成「变化通知」:只有新入榜、新新闻、编辑推荐时才占版面;榜全稳且无新新闻时静默不推。读者打开企微即知「今天值得扫一眼的是什么」。
## Explicit Non-Goals已否决方案
以下方案**不在本设计范围内**
| 方案 | 状态 |
|------|------|
| 静态页 / 外链档案库 | ❌ 不做 |
| 今日一装(每天一个 `npx skills add` | ❌ 不做 |
| 按星期轮换版面 | ❌ 不做 |
| 榜首锚点(稳定日仍展示 #1 | ❌ 不做 |
## Premises
1. 重复感主要来自**列表区块全量复印**,而非 opening 里引用榜首数字。
2. `output/*.data.json``daily/delta.py` 已具备新入榜对比能力,应上升为**列表渲染主数据源**。
3. 企微消息仍在应用内读完,不依赖外部页面。
4. 叙事层opening、信号、首推、新闻保持充实缩短的是**榜单列表**,不是整报。
## Recommended Approach: Delta 模式
### 环境变量
```env
# full = 现有行为(全量 Top 榜列表)
# delta = 本设计(默认推荐)
DAILY_WECOM_MODE=delta
# 无对比基准时(首日或缺历史 data.json是否自动 full 一次
DAILY_DELTA_BASELINE_FALLBACK=full # full | empty
# 推送闸门全不满足时是否跳过 webhook仍写 output 文件)
DAILY_SKIP_PUSH_WHEN_SILENT=1
# 强制推送(忽略静默)
# DAILY_FORCE_PUSH=1
# 新闻:缩短窗口 + 去重天数
DAILY_AI_NEWS_HOURS=24
DAILY_NEWS_DEDUP_DAYS=7
```
### 推送闸门Push Gate
满足**任一**条件则生成并推送企微早报:
| 条件 | 数据源 |
|------|--------|
| 任榜单有新入条目 | `movement.*_moves` 非空 |
| 去重后仍有新新闻 | 国际或国内 AI 时讯 |
| 存在 `featured_pick` | Step 0 编辑推荐 |
| `DAILY_FORCE_PUSH=1` | 环境变量 |
**静默日**:以上皆不满足 → 不调用 webhook`DAILY_SKIP_PUSH_WHEN_SILENT=1` 时)。
仍执行 `daily generate`,写入 `output/{date}.md``output/{date}.wecom.md``output/{date}.data.json` 留档。
**注意**:仅新闻有新、榜单全稳时**仍推送**,但 Skills/GitHub 列表区块整块省略(不是全天静默)。
### 列表渲染Delta 列表)
`DAILY_WECOM_MODE=delta` 时:
#### Skills
- **仅展示** `movement.skills_trending_moves` / `movement.skills_hot_moves` 中的新入榜条目。
- **跨榜去重**:按 `skill_id``id``source/title`)合并;同一 skill 只出现一次,标注来源榜(如 `Trending #4 · Hot #2`)。
- **无新入**:该榜区块**整块不出现**(不写多行「较昨日无新增」)。
#### GitHub
- 仅展示 `movement.github_trending_moves``github_emerging_moves``github_topic_moves`
- 无新入则区块省略。
#### 不包含
- 全量 Top N 列表
- 榜首锚点
- `(新入 … #n` 括号标注(与现 `agent_workflow._strip_new_entry_notes` 一致,列表标题用 `[新入 #n]` 前缀即可)
### 固定骨架(不因 Delta 缩短)
Agent 模式(`DAILY_REPORT_MODE=agent`)下,以下区块**保持**
- opening23 句,首句含具体证据)
- headline / 今日主题
- 今日信号35 条)
- 今日首推
- 国际 AI / 国内 AI 精选(条数仍由 `DAILY_WECOM_AI_NEWS` 等控制)
榜单变短;叙事与新闻不主动砍到 0。
### Full 模式逃生口
`DAILY_WECOM_MODE=full` 时行为与**当前生产一致**`format_wecom.build_wecom_report` / `replace_wecom_skill_sections` 全量 Top N。用于手动切回或对比测试。
### 首日 / 无历史基准
`find_previous_data(date)` 返回 `None` 时:
| `DAILY_DELTA_BASELINE_FALLBACK` | 行为 |
|----------------------------------|------|
| `full`(推荐) | 当日按 full 模式渲染列表一次;次日起 delta |
| `empty` | 当日列表区块为空opening 须说明「首日报,暂无对比基准」 |
实现时在 `generate_report``build_llm_input` 传入 `baseline_date` 供 Agent 引用。
## News Dedup
### P0本阶段
- 维护 `cache/pushed-news-links.json`(或写入 `output/` 旁 cache最近 `DAILY_NEWS_DEDUP_DAYS` 天已出现在企微早报中的 `link` 集合。
- `prepare_wecom_news_items` / `prepare_wecom_cn_news_items` 输出前过滤已见 link。
- `DAILY_AI_NEWS_HOURS` 默认改为 `24``.env.example` 同步)。
### P1可选后续
- 标题归一化去重(同一事件多源报道)
-`source_name` 每日上限 N 条
## Agent Skill 变更
文件:`skills/daily-agent/SKILL.md`
### 删除 / 修改
- 删除规则:「即使某榜较昨日无新增,仍须完整列出 Top 榜条目」。
- 删除:「禁止改用 movement 作为列表来源」(在 delta 模式下反转)。
### 新增
`DAILY_WECOM_MODE=delta`(或 llm_input 含 `wecom_mode: delta`
1. Agent **不写** Skills Trending / Hot / GitHub 列表(仍由 Python 插入,与现流程一致)。
2. opening / signals **可引用**榜首与 movement 摘要;禁止在 signals 重复列表已展示的同一事实。
3. 榜全稳时signals 聚焦新闻与首推,不必编造榜单变化。
`wecom_mode: full` 时保持现有 SKILL 规则。
## Python 模块变更
| 模块 | 变更 |
|------|------|
| `daily/config.py` | `wecom_mode()`, `news_dedup_days()`, `skip_push_when_silent()`, `delta_baseline_fallback()` |
| `daily/delta.py` | 可选:`merge_skill_moves_for_wecom(trending_moves, hot_moves)` 跨榜去重 |
| `daily/format_wecom.py` | `build_skills_delta_section()`, `build_github_delta_section()``replace_wecom_skill_sections` 支持 delta |
| `daily/news/fetch.py` | `filter_pushed_news()` + cache 读写 |
| `daily/generate.py` | 推送闸门baseline fallback静默 skip push |
| `daily/report_data.py` | `llm_input` 增加 `wecom_mode`, `push_gate` 摘要 |
| `daily/agent_workflow.py` | 无逻辑变更;依赖 Python 插入 delta 列表 |
| `.env.example` | 新 env 文档 |
## 企微消息示例
### 有变化日
```markdown
📰 **早报 · 2026-07-10**
[opening今天最大变化含数字/条目名]
🎯 **{headline}**
💡 **今日信号**
> ...
📦 **今日首推**
[...]
🌍 **国际 AI · 精选 N**
...
📈 **Skills Trending 变化**
1. [新入 #4] [**xxx**](...) · ...
描述一行
🔥 **Skills Hot 变化**
1. [新入 #2] [**yyy**](...) · ...
🐙 **GitHub Trending 变化**
1. [新入 #4] [owner/repo](...) · ...
```
### 仅新闻有新(榜稳)
- 无 📈/🔥/🐙 区块
- opening 可一句:「榜单较昨日 Top15 无新入;以下为今日 AI 时讯。」
### 静默日
- 不推送企微
- `output/` 仍落盘;日志:`[silent] no push gate matched for 2026-07-10`
## Approaches Considered
### Approach A: 配置瘦身(缩 Top N、24h 新闻)
- Effort: S | Risk: Low
- 只减篇幅,榜头仍天天重复;未解决根因。
### Approach B: Delta 列表 + 推送闸门 + 新闻去重(本设计)
- Effort: M | Risk: Med
- 复用 `delta.py`;改 format + skill + push 逻辑。
### Approach C: 仅改 Agent 文案
- Effort: S | Risk: High
- Python 仍插入全量列表,规则冲突,不可持续。
**Recommendation: B** — 数据层与展示层一致,静默日与跨榜去重可测。
## Success Criteria
1. 连续 3 天对比 `output/*.data.json`:企微列表区块**重复 skill_id 占比**显著下降。
2. 榜全稳且无新新闻日:`DAILY_SKIP_PUSH_WHEN_SILENT=1` 时不发 webhook。
3. Trending/Hot 同一 skill 在列表中**最多出现 1 次**。
4. `DAILY_WECOM_MODE=full` 与现网行为一致(回归用)。
5. 首日 `DAILY_DELTA_BASELINE_FALLBACK=full` 不产生空列表投诉。
## Open Questions
1. 静默日是否需要在企微发一行「今日无更新」?当前设计:**不发**。
2. 新闻去重 cache 是否纳入 git建议**否**,放 `cache/`(已在 `.gitignore`)。
3. Classic 模式(非 agent是否同步 delta建议**是**,同一 `format_wecom` 路径。
## Implementation Tasks
| ID | Priority | Task | Files |
|----|----------|------|-------|
| T1 | P1 | 新增 config helpers + `.env.example` | `daily/config.py`, `.env.example` |
| T2 | P1 | 新闻 link 去重 cache | `daily/news/fetch.py`, `daily/config.py` |
| T3 | P1 | Delta 列表渲染 + 跨榜去重 | `daily/format_wecom.py`, `daily/delta.py` |
| T4 | P1 | 推送闸门 + 静默 skip push | `daily/generate.py`, `daily/webhook.py` |
| T5 | P1 | baseline fallback full 一次 | `daily/generate.py` |
| T6 | P1 | 更新 `daily-agent/SKILL.md` | `skills/daily-agent/SKILL.md` |
| T7 | P2 | `llm_input``wecom_mode` / push 摘要 | `daily/report_data.py` |
| T8 | P2 | 单元测试:跨榜去重、推送闸门、新闻去重 | `tests/test_wecom_delta.py` |
## Test Plan
- [ ]`2026-07-09.data.json` 时生成 `2026-07-10`:列表仅含新入项
- [ ] 人造「全稳 + 无新新闻」:不 push
- [ ] 人造「全稳 + 有新新闻」push无 Skills/GitHub 块
- [ ] `DAILY_WECOM_MODE=full` 输出与改前 `2026-07-09.wecom.md` 结构一致
- [ ] 无 baseline + `DAILY_DELTA_BASELINE_FALLBACK=full`:首日全量列表
- [ ] 同一 skill 在 Trending/Hot moves 均出现:列表只 1 条
## What I Noticed
- 重复感是**产品形态**问题,不是 Agent 文笔问题。
- 明确否决静态页、今日一装、轮换、锚点后,方案边界清晰,实现可分期。
- 推送闸门必须**把新闻算进去**,否则静默日会被新闻绕过。

View File

@@ -1,35 +0,0 @@
<!-- 示例Agent 模式企微早报节选,日期 2026-07-03 -->
📰 **早报 · 2026-07-03**
> ⏱ 09:30 (UTC+8) · 数据截至 2026-07-02
> remotion-render 以 21549 次安装稳坐 Skills Trending 榜首halt-catch-fire 五件套把「TSX 直出 MP4」推到台前。GitHub 侧 openclaw 仍以 381.5K star 居 Trending 第一;新闻头条则是 OpenAI 拟向特朗普政府让渡 AI 繁荣 5% 分成——多媒体 skill 装机、本地 Agent 开源栈与 AI 治理议题同日撞屏。
🎯 **代码出片霸榜Agent基建共振**
💡 **今日信号**
> 🔥 Skills Hotxixu-me/skills 12 件套新占 Top10 前十,含 xdrop、openclaw-secure-linux-cloud 等开发者工具簇
> 📦 Skills较昨日 Trending Top15 新入 7 条,飞书 lark-approval、lark-wiki 等办公 skill 批量涌入
> 🐙 GitHubaffaan-m/ECC 225.2K star 新入 Trending #2面向 Claude Code/Codex/Cursor 的 Agent harness 优化成显学
> 🌱 GitHublangflow、dify 新入 Trending Top10可视化 Agent 工作流平台与 DeepSpec 推测解码基建各据一角
> 📰 时讯Anthropic 搁置已久的 Fable 5 获准回归,模型内容政策与 Zuckerberg「Agent 进展慢于预期」形成对照
📦 **今日首推**
`npx skills add halt-catch-fire/skills/remotion-render`
> Trending #1、21549 安装5 skill 集群覆盖 remotion-render 到 ai-video-generation把 React/Remotion 组件程序化渲染 MP4 做成默认可装工作流。
🌍 **国际 AI · 精选 10**
1. [OpenAI拟向特朗普政府让渡AI繁荣5%分成](https://www.theverge.com/ai-artificial-intelligence/960588/openai-government-5-percent-stake-trump) — The Verge 头条:头部模型公司探讨向政府让渡 AI 收益分成,商业化与治理边界被推上舆论前台
2. [Anthropic搁置已久的Fable 5获准回归](https://www.theverge.com/ai-artificial-intelligence/958964/anthropic-claude-fable-5-is-back) — 经数周协商后,此前被搁置的 Claude Fable 5 内容政策模型重新获准上线
3. [Zuckerberg称AI Agent进展慢于预期](https://techcrunch.com/2026/07/02/mark-zuckerberg-tells-staff-that-ai-agents-havent-progressed-as-quickly-as-hed-hoped/) — Meta 内部会议透露Agent 落地节奏未达此前预期
🇨🇳 **国内 AI · 精选 8**
1. [让Agent越用越强AReaL 2.0开源打造面向自演进智能体的RL基础设施](https://www.qbitai.com/2026/07/442134.html) — 量子位:面向自演进 Agent 的 RL 训练基建开源
2. [Agent Loop 深度调研:把决定权交给模型的一次换代,为什么发生在现在](https://juejin.cn/post/7657771483503018036) — 掘金长文梳理 Agent Loop 架构演进
📈 **Skills Trending Top 9**
1. [**halt-catch-fire/skills** · 5 skills · **21.5K21.5K**](https://www.skills.sh/halt-catch-fire/skills/remotion-render)
> 用 inference.sh 将 Remotion 组件代码直接渲染为 MP4可配分辨率帧率时长适合程序化视
2. [**find-skills**](https://www.skills.sh/vercel-labs/skills/find-skills) · `vercel-labs/skills` · **18.0K**
> 用户询问如何做某事或寻找能力扩展时,帮助发现并安装合适的 Agent Skill。
<!-- 完整版还包含 Hot 榜、GitHub Trending/新兴/Topic 等分区,见本地 output/ -->

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,604 @@
# 企微早报多样性与去重 Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** 实现企微早报硬去重多样性:五榜周去重(深池补满)、首推与昨日相同则改推(月去重)、叙事轴代码互斥、取消新闻「放宽窗口」凑数;且 `movement_baseline``wecom_shown_keys` 严格分离。
**Architecture:**`daily generate` 管线加代码选择器:`board_select` 为 full/delta 唯一列表主人;周历史只读写 `data.wecom_shown_keys`post-render`movement_baseline` 仍为 raw Top compare`featured_resolve` 先定人再 research`pick_narrative_axis` 代码选轴注入 Agent Step1新闻关 backfill + 剥「放宽」前缀。
**Tech Stack:** Python 3.13+、现有 `unittest`/`pytest``daily/delta.py` / `format_wecom.py` / `featured_pick.py` / `news/fetch.py``output/*.data.json`
**Spec:** `docs/superpowers/specs/2026-07-14-wecom-diversity-dedup-design.md`Status: APPROVED
## Global Constraints
- `movement_baseline` **禁止**被展示历史覆写;周去重只读 `wecom_shown_keys`
- full/delta **唯一列表主人** = `board_select`delta 的 pad 共用同一套 shown 历史)
- `DAILY_BOARD_DEDUP_DAYS` 默认 `7`;与 pad lookback 对齐且数据源同一
- `DAILY_FEATURED_DEDUP_DAYS` 默认 `30`
- `DAILY_NARRATIVE_AXIS_DAYS` 默认 `3`;轴枚举固定 7 个(见 Task 5
- `DAILY_NEWS_BACKFILL` 默认 `0`(禁止旧闻凑数)
- research 补新闻年龄上限 = `DAILY_AI_NEWS_HOURS`(不得变相 48h 放宽)
- 不做:语义「同一类」、同日 Trending↔Hot 互斥、`FEATURED_FORCE`、关键短语硬匹配
- Commit 信息正文用中文(若本任务含 Commit 步)
---
## File Structure
| 文件 | 职责 |
|------|------|
| `daily/config.py` | 新 env`board_dedup_days``board_pool_size``featured_dedup_days``theme_ban_days``narrative_axis_days``news_backfill_enabled` |
| `daily/board_history.py` | **新建**`load_recent_shown_keys` / `extract_shown_keys` / `attach_wecom_shown_keys`(读写 `data.wecom_shown_keys` |
| `daily/board_select.py` | **新建**`board_select(...)` 周过滤+深池 |
| `daily/delta.py` | `load_recent_board_keys` 改为委托 `load_recent_shown_keys`(保留函数名兼容);**不**改 `build_movement_baseline` |
| `daily/format_wecom.py` | pad 使用 shown keys可选返回最终展示 items 供写回 |
| `daily/featured_pick.py` | `featured_identity_key``featured_resolve`、先定人再 research |
| `daily/narrative_axis.py` | **新建**`NARRATIVE_AXES``pick_narrative_axis``load_recent_axes` |
| `daily/news/fetch.py` | `_apply_pushed_dedup_with_backfill` 尊重 `news_backfill_enabled()`;默认不塞回 |
| `daily/news/research.py` + `skills/daily-ai-news-research/SKILL.md` | 禁放宽文案;补入不超时窗 |
| `daily/text_utils.py``daily/news/sanitize.py` | `strip_news_relax_prefix(desc)` |
| `daily/agent_workflow.py` | Step1 注入 axis + 近 7 日 theme 软禁;强制覆写冲突轴 |
| `daily/generate.py` | 串联select → featured → editorial → render → persist shown/key/axis |
| `daily/report_data.py` | data.json 可携 `wecom_shown_keys` / `featured_pick_key` / `narrative_axis`(写回可由 generate 合并) |
| `.env.example` | 文档化新变量 |
| `skills/daily-agent/SKILL.md` | `narrative_axis` 必填且等于输入指定轴 |
| `tests/test_board_select.py` | **新建** |
| `tests/test_board_history.py` | **新建** |
| `tests/test_featured_resolve.py` | **新建** |
| `tests/test_narrative_axis.py` | **新建** |
| `tests/test_news_relax.py` | **新建** |
| `tests/test_wecom_delta.py` | 回归pad 不读 movement 当展示史 |
---
### Task 1: Config + shown-keys 历史层
**Files:**
- Modify: `daily/config.py`(文件末尾追加)
- Create: `daily/board_history.py`
- Modify: `daily/delta.py``load_recent_board_keys` 改委托)
- Test: `tests/test_board_history.py`
- Modify: `.env.example`
**Interfaces:**
- Consumes: `OUTPUT_DIR`、现有 `delta.skill_id` / repo key 约定
- Produces:
- `board_dedup_days() -> int`(默认 7
- `board_pool_size() -> int`(默认 `max(50, env WECOM_SKILL_POOL)`
- `featured_dedup_days() -> int`(默认 30
- `theme_ban_days() -> int`(默认 7
- `narrative_axis_days() -> int`(默认 3
- `news_backfill_enabled() -> bool`(默认 Falseenv `DAILY_NEWS_BACKFILL`
- `extract_shown_keys(board: str, items: list[dict]) -> list[str]`
- `load_recent_shown_keys(date_str: str, *, lookback_days: int | None = None) -> dict[str, set[str]]`
- `merge_wecom_shown_into_data(data: dict, shown: dict[str, list[str]]) -> dict`
- [ ] **Step 1: Write the failing test**
```python
# tests/test_board_history.py
from __future__ import annotations
import json
import os
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from daily.board_history import extract_shown_keys, load_recent_shown_keys, merge_wecom_shown_into_data
from daily.config import board_dedup_days, news_backfill_enabled
class ConfigDiversityTests(unittest.TestCase):
def test_board_dedup_days_default(self):
with patch.dict(os.environ, {}, clear=True):
self.assertEqual(board_dedup_days(), 7)
def test_news_backfill_default_off(self):
with patch.dict(os.environ, {}, clear=True):
self.assertFalse(news_backfill_enabled())
class ShownKeysTests(unittest.TestCase):
def test_extract_github_repo_keys(self):
items = [{"repo": "a/b"}, {"repo": "c/d"}]
self.assertEqual(extract_shown_keys("github_trending", items), ["a/b", "c/d"])
def test_load_recent_reads_wecom_shown_not_baseline(self):
with tempfile.TemporaryDirectory() as tmp:
out = Path(tmp)
# 前日shown 只有 x/ybaseline raw 含 a/b —— 周去重只能看到 x/y
payload = {
"data": {
"date": "2026-07-13",
"movement_baseline": {
"github_trending": [{"repo": "a/b"}, {"repo": "x/y"}],
},
"wecom_shown_keys": {"github_trending": ["x/y"]},
}
}
(out / "2026-07-13.data.json").write_text(
json.dumps(payload, ensure_ascii=False), encoding="utf-8"
)
with patch("daily.board_history.OUTPUT_DIR", out):
keys = load_recent_shown_keys("2026-07-14", lookback_days=7)
self.assertEqual(keys["github_trending"], {"x/y"})
self.assertNotIn("a/b", keys["github_trending"])
def test_merge_shown_does_not_touch_baseline(self):
data = {
"movement_baseline": {"github_trending": [{"repo": "raw/one"}]},
}
merged = merge_wecom_shown_into_data(
data, {"github_trending": ["shown/one"]}
)
self.assertEqual(
merged["movement_baseline"]["github_trending"][0]["repo"], "raw/one"
)
self.assertEqual(merged["wecom_shown_keys"]["github_trending"], ["shown/one"])
```
- [ ] **Step 2: Run test to verify it fails**
Run: `pytest tests/test_board_history.py -v`
Expected: FAIL模块/函数不存在)
- [ ] **Step 3: Implement config + board_history + delta 委托**
`daily/config.py` 追加:
```python
def board_dedup_days() -> int:
return max(1, env_int("DAILY_BOARD_DEDUP_DAYS", 7))
def board_pool_size() -> int:
fallback = env_int("DAILY_WECOM_SKILL_POOL", 50)
return max(1, env_int("DAILY_BOARD_POOL_SIZE", max(50, fallback)))
def featured_dedup_days() -> int:
return max(1, env_int("DAILY_FEATURED_DEDUP_DAYS", 30))
def theme_ban_days() -> int:
return max(1, env_int("DAILY_THEME_BAN_DAYS", 7))
def narrative_axis_days() -> int:
return max(1, env_int("DAILY_NARRATIVE_AXIS_DAYS", 3))
def news_backfill_enabled() -> bool:
return env_bool("DAILY_NEWS_BACKFILL", False)
```
新建 `daily/board_history.py`:实现 `BOARD_KEYS``delta.RECENT_BOARD_KEYS` 同五榜Skills 用 `delta.skill_id`GitHub 用 `repo``load_recent_shown_keys` **只**读各日 `data.wecom_shown_keys`,缺省空集,读写失败打 log 后当空集。
修改 `daily/delta.py``load_recent_board_keys`:改为
```python
def load_recent_board_keys(date_str: str, *, lookback_days: int | None = None) -> dict[str, set[str]]:
from daily.board_history import load_recent_shown_keys
from daily.config import board_dedup_days
days = lookback_days if lookback_days is not None else board_dedup_days()
return load_recent_shown_keys(date_str, lookback_days=days)
```
删除(或不再走)原「从 movement_baseline 抽 keys」逻辑避免 pad 继续把 raw Top 当展示史。
`.env.example` 追加注释块:
```env
# 多样性 / 去重(见 docs/superpowers/specs/2026-07-14-wecom-diversity-dedup-design.md
# DAILY_BOARD_DEDUP_DAYS=7
# DAILY_BOARD_POOL_SIZE=50
# DAILY_FEATURED_DEDUP_DAYS=30
# DAILY_THEME_BAN_DAYS=7
# DAILY_NARRATIVE_AXIS_DAYS=3
DAILY_NEWS_BACKFILL=0
```
- [ ] **Step 4: Run tests**
Run: `pytest tests/test_board_history.py tests/test_wecom_delta.py -v`
Expected: `test_board_history` PASS既有 delta 测试若依赖「baseline 即 recent」行为按 Task 1 语义改断言为 shown_keys本 Task 内修回归,勿留红)。
- [ ] **Step 5: Commit**
```bash
git add daily/config.py daily/board_history.py daily/delta.py .env.example tests/test_board_history.py tests/test_wecom_delta.py
git commit -m "feat: 拆分 wecom_shown_keys 与 movement_baseline 历史层"
```
---
### Task 2: `board_select` 周去重 + 深池
**Files:**
- Create: `daily/board_select.py`
- Test: `tests/test_board_select.py`
**Interfaces:**
- Consumes: `extract_shown_keys` / `skill_id``group_skills_by_source`Skills 板)
- Produces:
- `board_select(*, board: str, items: list[dict], recent_keys: set[str], limit: int, pool_size: int, kind: Literal["skill","github"]) -> list[dict]`
- 日志短榜:`board_short:{board}:{n}``logging.getLogger(__name__).info`
- [ ] **Step 1: Write the failing test**
```python
# tests/test_board_select.py
from __future__ import annotations
import unittest
from daily.board_select import board_select
def _gh(repo: str) -> dict:
return {"repo": repo, "description": repo}
class BoardSelectTests(unittest.TestCase):
def test_filters_recent_and_keeps_order(self):
pool = [_gh(f"o/r{i}") for i in range(20)]
recent = {"o/r0", "o/r1", "o/r2"}
out = board_select(
board="github_trending",
items=pool,
recent_keys=recent,
limit=5,
pool_size=20,
kind="github",
)
keys = [x["repo"] for x in out]
self.assertEqual(keys, ["o/r3", "o/r4", "o/r5", "o/r6", "o/r7"])
def test_deep_pool_fills_after_filter(self):
pool = [_gh(f"o/r{i}") for i in range(8)]
recent = {f"o/r{i}" for i in range(6)} # 前 6 全封
out = board_select(
board="github_emerging",
items=pool,
recent_keys=recent,
limit=5,
pool_size=8,
kind="github",
)
self.assertEqual([x["repo"] for x in out], ["o/r6", "o/r7"]) # 短榜
def test_skill_uses_skill_id(self):
items = [
{"id": "a/b/s1", "source": "a/b", "title": "s1", "installs": 10},
{"id": "c/d/s2", "source": "c/d", "title": "s2", "installs": 9},
]
out = board_select(
board="skills_trending",
items=items,
recent_keys={"a/b/s1"},
limit=10,
pool_size=50,
kind="skill",
)
self.assertEqual([x["id"] for x in out], ["c/d/s2"])
```
- [ ] **Step 2: Run test to verify it fails**
Run: `pytest tests/test_board_select.py -v`
Expected: FAIL
- [ ] **Step 3: Implement `board_select`**
```python
# daily/board_select.py — 核心逻辑示意
def board_select(*, board, items, recent_keys, limit, pool_size, kind):
if kind == "skill":
from daily.skills_group import group_skills_by_source
pool = group_skills_by_source(items, limit=pool_size, pool_size=pool_size)
def key_fn(x): return skill_id(x)
else:
pool = items[: max(pool_size, limit)]
def key_fn(x): return str(x.get("repo") or "")
out = []
for item in pool:
k = key_fn(item)
if not k or k in recent_keys:
continue
out.append(item)
if len(out) >= limit:
break
if len(out) < limit:
logger.info("board_short:%s:%s", board, len(out))
return out
```
Skills输入可为未 group 的 raw函数内 group。GitHub输入为 repo 列表。
- [ ] **Step 4: Run tests — expect PASS**
Run: `pytest tests/test_board_select.py -v`
- [ ] **Step 5: Commit**
```bash
git add daily/board_select.py tests/test_board_select.py
git commit -m "feat: 实现 board_select 周去重与深池补满"
```
---
### Task 3: 接入 generate / format_wecom唯一列表主人 + post-render 写回)
**Files:**
- Modify: `daily/generate.py`(选榜、传入 pad、渲染后 merge shown
- Modify: `daily/format_wecom.py`delta pad 已通过改写后的 `load_recent_board_keys` 读 shown确保传入的 `*_pad` 池已是 `board_select` 深池结果)
- Modify: `daily/report_data.py`可选llm_input 切片改为 board_select 后列表,避免 Agent 看见未去重 Top
- Test: `tests/test_board_history.py` 增补「shown ≠ baseline 推导」集成断言;`tests/test_wecom_delta.py` pad 用例
**Interfaces:**
- Consumes: Task12
- Produces: 每次成功 generate 后 `output/{date}.data.json``data.wecom_shown_keys`
- [ ] **Step 1: Write / extend failing integration test**
```python
def test_persist_shown_keys_differs_from_baseline_keys(self):
# 构造raw trending 前 3 名本周已 shownboard_select 选出 3..
# movement_baseline 仍含 0..compare_depth
# 断言 data["wecom_shown_keys"]["github_trending"] 与 baseline repos 集合不等
...
```
(可用临时 `OUTPUT_DIR` + 调用抽取出的 `persist` 辅助,或测 `merge_wecom_shown_into_data` + `board_select` 组合。)
- [ ] **Step 2: Run — expect FAILgenerate 尚未写 shown**
- [ ] **Step 3: Wire generate**
`generate_report` 中,在组装 wecom 榜之前:
1. `recent = load_recent_shown_keys(date_str)`
2. 对五榜分别 `board_select(...)` 得到 `selected_*`limit=wecom_*pool=`board_pool_size()`
3. full渲染用 `selected_*`
4. delta`trending_pad`/`github_*_pad` = 同规则更大 pool 的 select 结果(或 raw 深池再 select`replace_wecom_board_sections(..., pad=True)` 内部 recent 已是 shown
5. 渲染后根据**最终写入正文的 items**full=selecteddelta=函数返回或并行计算最终列表)调用 `extract_shown_keys``merge_wecom_shown_into_data`,写回 data.json在现有 `save_json` 路径合并字段)
注意:`movement_baseline` 仍用 **raw** compare 切片构建(`report_data.build_llm_input` 现逻辑保留)。
`build_llm_input` 当前把未过滤 Top 塞进 Agent改为传入 `selected_*`(或另字段 `boards_for_wecom`),避免 opening 引用已周封杀的榜首。
辅助:在 `format_wecom` 增加 `resolve_wecom_board_items(...)` 返回最终 items dict供写回与 featured 池 A 共用,避免正文与 history 分叉。
- [ ] **Step 4: Run tests**
Run: `pytest tests/test_board_select.py tests/test_board_history.py tests/test_wecom_delta.py -v`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add daily/generate.py daily/format_wecom.py daily/report_data.py tests/
git commit -m "feat: generate 以 board_select 为唯一列表主人并写回 shown keys"
```
---
### Task 4: `featured_resolve`(先定人再 research
**Files:**
- Modify: `daily/featured_pick.py`
- Modify: `daily/generate.py`(调用顺序)
- Test: `tests/test_featured_resolve.py`
**Interfaces:**
- Consumes: 最终展示 items池 A、raw 深池(池 B、近 30 日 `featured_pick_key`
- Produces:
- `featured_identity_key(featured: dict) -> str``type==skill` → idgithub → repo兜底 url path
- `load_recent_featured_keys(date_str, days) -> set[str]`
- `featured_resolve(*, date_str, candidate: dict | None, pool_a: list[dict], pool_b: list[dict], rng: random.Random | None) -> tuple[dict | None, str | None]`
返回 `(resolved_seed_or_featured_stub, identity_key)`**不含**完整 whywhy 由后续 research 写)
-`apply_featured_pick`:先 resolve 身份(若与昨日冲突则换候选写入 query`research_featured_pick`
- [ ] **Step 1: Failing tests**
```python
# tests/test_featured_resolve.py
def test_same_as_yesterday_picks_from_pool_a(self):
yesterday_key = "headroomlabs-ai/headroom"
pool_a = [
{"repo": "headroomlabs-ai/headroom", "board": "github_topic"},
{"repo": "ollama/ollama", "board": "github_trending"},
]
rng = random.Random(0)
resolved, key = featured_resolve(
date_str="2026-07-14",
candidate={"type": "github", "url": "https://github.com/headroomlabs-ai/headroom", "title": "headroom"},
pool_a=pool_a,
pool_b=[],
recent_featured={yesterday_key},
yesterday_key=yesterday_key,
rng=rng,
)
self.assertNotEqual(key, yesterday_key)
self.assertEqual(key, "ollama/ollama")
def test_pool_a_before_pool_b(self):
...
def test_exhausted_keeps_original(self):
...
```
- [ ] **Step 2: Run — FAIL**
- [ ] **Step 3: Implement**
`featured_resolve`:若无 candidate 或与 `yesterday_key` 不同 → 原样返回。
冲突时:过滤 `recent_featured | {yesterday_key}`,先从 pool_a 建可选项(每项抽 identity`rng.choice`;空则 pool_b仍空 log `featured_fallback_exhausted` 并保留原 candidate。
`apply_featured_pick` / generate 流程:
1. 解析 env 得到初始 query/candidate
2. `featured_resolve`(此时池 A 已是 board_select 结果)
3. 若换人:用新 repo/skill 构造 config`research_featured_pick`
4. 写入 `llm_input["featured_pick"]` 与之后 data.`featured_pick_key`
随机默认:`random.Random(int(hashlib.sha256(f"{date_str}:featured".encode()).hexdigest()[:16], 16))`
- [ ] **Step 4: pytest PASS**
- [ ] **Step 5: Commit**
```bash
git add daily/featured_pick.py daily/generate.py tests/test_featured_resolve.py
git commit -m "feat: 首推与昨日冲突时改推并保证一月不重复"
```
---
### Task 5: `narrative_axis` 硬互斥 + Step1 软禁 theme
**Files:**
- Create: `daily/narrative_axis.py`
- Modify: `daily/agent_workflow.py``analyze_trends`
- Modify: `skills/daily-agent/SKILL.md`
- Modify: `daily/generate.py`(落盘 `narrative_axis`;注入 llm_input
- Test: `tests/test_narrative_axis.py`
**Interfaces:**
- Produces:
- `NARRATIVE_AXES: tuple[str, ...] = ("政策监管", "模型发布", "工具链/Agent", "芯片算力", "开源生态", "应用落地", "安全/诉讼")`
- `pick_narrative_axis(used: set[str], *, rng: random.Random | None = None) -> str`
- `load_recent_axes(date_str, days) -> list[str]`(近 N 日 data.`narrative_axis`
- `enforce_narrative_axis(trends: dict, axis: str) -> dict`(强制 trends["narrative_axis"]=axis
- [ ] **Step 1: Failing tests**
```python
def test_pick_excludes_used(self):
used = {"政策监管", "模型发布", "工具链/Agent"}
for _ in range(20):
axis = pick_narrative_axis(used, rng=random.Random(1))
self.assertNotIn(axis, used)
def test_enforce_overwrites_llm(self):
trends = {"narrative_axis": "开源生态", "opening": "..."}
out = enforce_narrative_axis(trends, "芯片算力")
self.assertEqual(out["narrative_axis"], "芯片算力")
```
- [ ] **Step 2: FAIL → Step 3 implement**
`analyze_trends`:计算 `axis = pick_narrative_axis(set(load_recent_axes(...)))`;把 `required_narrative_axis` 与近 `theme_ban_days` 的 theme/opening 摘要列表注入 system prompt要求 JSON 含 `narrative_axis` 且必须等于 required。解析后 `enforce_narrative_axis`
generate 将 axis 写入 data.json。
SKILL.md Step1 schema 增加 `narrative_axis` 字段说明。
- [ ] **Step 45: pytest + commit**
```bash
git commit -m "feat: 代码选定叙事轴并注入 Agent 开场约束"
```
---
### Task 6: 取消新闻「放宽窗口」
**Files:**
- Modify: `daily/news/fetch.py``_apply_pushed_dedup_with_backfill`
- Modify: `daily/news/research.py`(确认只 filter_unpushed不足不拉超窗
- Create: `daily/news/sanitize.py`(或放入 `text_utils`)— `strip_relax_window_prefix(text: str) -> str`
- Modify: `skills/daily-ai-news-research/SKILL.md`(删除「放宽至 48h 并注明」;改为不足则少返回、禁止标注)
- Modify: `daily/generate.py` / news finalize对 desc_short 剥前缀)
- Test: `tests/test_news_relax.py`;扩展 `tests/test_news_fetch_window.py`
**Interfaces:**
- `news_backfill_enabled() == False` 时:`_apply_pushed_dedup_with_backfill` 等价于只返回 `filter_unpushed_items(...)[:limit]`**不**再从 `picked` 塞回
- `strip_relax_window_prefix`:去掉开头的 `放宽窗口[:]?` / `放宽至[^:]*[:]`
- [ ] **Step 1: Failing tests**
```python
def test_backfill_disabled_does_not_reinsert_pushed(self):
# fresh 不足 limitpicked 含已推BACKFILL=0 → 结果不含已推 link
...
def test_strip_relax_prefix(self):
self.assertEqual(
strip_relax_window_prefix("放宽窗口:苹果起诉 OpenAI"),
"苹果起诉 OpenAI",
)
```
- [ ] **Step 24: 实现并跑 `pytest tests/test_news_relax.py tests/test_news_fetch_window.py -v`**
Research 路径SKUILL 改完后,代码侧对 items 统一 `strip`;不足时 log `news_short:{n}`,接受短列表。
- [ ] **Step 5: Commit**
```bash
git commit -m "fix: 关闭新闻放宽凑数并剥离放宽窗口文案"
```
---
### Task 7: 端到端回归与文档对齐
**Files:**
- Modify: 如有遗漏的 `.env.example` / SKILL
- Test: 全量相关测试
- [ ] **Step 1: 跑全套**
Run:
```bash
pytest tests/test_board_history.py tests/test_board_select.py tests/test_featured_resolve.py tests/test_narrative_axis.py tests/test_news_relax.py tests/test_news_fetch_window.py tests/test_wecom_delta.py tests/test_featured_pick.py -v
```
Expected: 全部 PASS
- [ ] **Step 2: Spec 对照清单(人工)**
| Spec 要求 | 任务 |
|-----------|------|
| wecom_shown_keys ≠ movement_baseline | T1, T3 |
| board_select 唯一主人 + delta pad 共用 shown | T2, T3 |
| 首推月去重 A→B、先定人再 why | T4 |
| narrative_axis 硬保证 | T5 |
| 禁放宽 backfill + 剥前缀 + hours 窗 | T6 |
| 成功标准可测 | 各测覆盖 |
- [ ] **Step 3: Commit若有收尾文档**
```bash
git add -u
git commit -m "test: 多样性去重全链路回归通过"
```
---
## Spec Coverage Self-Review
| Spec 节 | 计划任务 |
|---------|----------|
| 1.1 双基准分离 | T1 |
| 1.2 唯一列表主人 | T2T3 |
| 1.3 真相表 | T1, T3T5 落盘字段 |
| 2.1 board_select | T2 |
| 2.2 featured_resolve | T4 |
| 3.1 narrative_axis + 软 theme | T5 |
| 3.2 取消放宽 | T6 |
| 4.x 配置/降级/测试 | T1T7 |
无 TBDCommit 信息均为中文描述体。
## Execution Handoff
Plan complete and saved to `docs/superpowers/plans/2026-07-14-wecom-diversity-dedup.md`.
**两种执行方式:**
1. **Subagent-Driven推荐** — 每任务新开子代理,任务间审查
2. **Inline Execution** — 本会话按 `executing-plans` 连续做完,设检查点
要哪个?

View File

@@ -0,0 +1,271 @@
# Design: 企微早报多样性与去重
Generated: 2026-07-14
Repo: daily-robots
Status: APPROVED
Mode: Builder
Related: `docs/design-wecom-delta-mode.md`Delta 列表模式)
Revision: office-hours A —— 拆分展示历史、单一列表主人、历史真相表2026-07-14
## Problem Statement
近两日企微早报(如 2026-07-13 / 07-14骨架相同
- **今日首推**连续两天同为 `headroom`
- **开场主题**同属「上下文压缩 + 视频/Skills」腔调
- **AI 时讯**出现「放宽窗口」旧闻凑数标注
- **Skills / GitHub 各榜**(尤其新兴榜)周内大量重复展示
读者需要「今天新信息」,而不是换日期的复印机。
## Decisions已确认
| 决策点 | 选择 |
|--------|------|
| 实现路径 | **A管线选择器**代码硬保证去重LLM 只写开场/理由/摘要;中文化仍走现有 `localize` |
| Skills「同一类」 | 暂不管;沿用现有 `group_skills_by_source`**已知残留**:同 source 换 skill id 仍可能周内再出现) |
| 周去重后不足 Top N | **深池补满**,仍保证周内未出现;池空则短榜,不破周约束 |
| 首推改推候选 | **先展示榜(池 A再 raw 深池(池 B**;一月内首推不重复 |
| 开场主题 | **近 7 天禁主题/句式(软)+ 叙事轴与近 3 天不同(硬,代码选轴)** |
| 取消「放宽窗口」 | **禁止旧闻/已推凑数**;不够则深度检索补新闻;禁止任何「放宽」文案标注;仍不够则短列表 |
| 展示历史 vs 异动基准 | **必须拆开**`movement_baseline``wecom_shown_keys` |
| full/delta 列表主人 | **唯一主人** = `board_select`(含 full 与 delta 的 movespad禁止二次独立选榜 |
## Explicit Non-Goals
| 项 | 状态 |
|----|------|
| 语义级 Skill「同一类」分类 | ❌ 本期不做 |
| 同日 Skills Trending ↔ Hot 互斥 | ❌ 本期不做 |
| 独立 editorial 微服务 | ❌ 不做 |
| 改写 `localize` 为脚本机翻 | ❌ 保持 LLM + 缓存 |
| 编辑指定首推豁免改推(`FEATURED_FORCE` | ❌ 本期不做 |
| 「开场关键短语」硬匹配算法 | ❌ 本期不做(仅 prompt 软约束;不进硬成功标准) |
## Recommended Approach: 管线选择器(路径 A
在现有 `daily generate` 内增加选条/裁决层,不新起进程:
```
采集 raw 榜 + 新闻
→ build movement_baselineraw Top compare_depth —— 仅供次日「新入榜」,禁止写展示历史)
→ board_select读 wecom_shown_keys 周历史;周去重 + 深池;输出当日最终展示列表)
· full直接取 board_select 结果前 N
· delta在 board_select 候选池内做 movespadpad 也只从该池/同规则深池取,不再另起一套历史)
→ featured_resolve与昨日首推相同则改推池 A = 本 run 最终展示 keys
→ research why先定人再写 why_today
→ news_select禁放宽凑数深检索补满剥「放宽」前缀
→ editorial代码选 narrative_axisprompt 附近 7 天 theme 软禁)
→ 渲染 wecom
→ 写回 wecom_shown_keys = 最终进入企微正文的榜条目 keyspost-render
→ 其余 history首推月、axis写入 data.json 约定字段
```
**LLM 负责**`opening` / `theme_line`、首推 `why_today`、新闻与榜单项中文摘要(`localize`)。
**代码负责**:谁上榜、首推换谁、周/月去重、`narrative_axis` 选取、是否允许旧闻。
### Approaches Considered
| | A 管线选择器(采用) | B 偏 LLM 约束 | C 独立 editorial 服务 |
|--|--|--|--|
| 优点 | 可测;与 pushed-links 模式一致 | 改 prompt 快 | 边界清晰 |
| 缺点 | 需动 generate / featured / news / format | 易漏、难测 | 过重 |
---
## Section 1 — 总览、列表主人、历史真相表
### 1.1 两种「基准」禁止混用
| 字段 | 含义 | 写入时机 | 读者 |
|------|------|----------|------|
| `movement_baseline` | **Raw** 各榜 Top `compare_depth`(现网语义不变) | `build_llm_input` / 采集后尽早 | `build_movement_context`(新入榜) |
| `wecom_shown_keys` | **读者实际见到**的各榜 key 集合(及可选 rank | **wecom 渲染完成之后** | `board_select` 周去重delta pad测试 |
**禁止**:把 `wecom_shown_keys` 写入或覆写 `movement_baseline`
**禁止**:让 `load_recent_board_keys` 继续读 `movement_baseline` 充当「已展示」——应改为读近 N 日 `wecom_shown_keys`(可保留函数名,换数据源;或新建 `load_recent_shown_keys`)。
### 1.2 唯一列表主人
`board_select`(模块可挂在 `daily/board_select.py` 或扩 `delta.py`)是各榜**最终展示行**的唯一生产者:
| 模式 | 行为 |
|------|------|
| `full` | `board_select(raw, shown_history) →` 至多 N 条,直接渲染 |
| `delta` | 先算相对 `movement_baseline` 的 moves展示 = `moves`(已在候选内)∪ `pad`**pad 候选必须来自同一周去重池**(与 full 同一套 `board_select` 规则),不得再读 raw baseline 当「已展示」 |
交互影响(非「完全正交」):周去重会减少可展示重复项 → delta 日可能更短、silent/gate 行为可能变化。`DAILY_WECOM_MODE` 枚举语义不变,但列表密度会变。
### 1.3 历史真相表(单一来源)
全部落在 `output/{date}.data.json`(新闻 pushed-links 例外,沿用现网 cache
| 字段路径 | 窗口 | Key 规则 | 写者 | 读者 |
|----------|------|----------|------|------|
| `data.movement_baseline` | 次日对比用 | raw 条目切片 | `build_movement_baseline` | movement |
| `data.wecom_shown_keys.{board}` | 滚动 7 天(读近 7 日文件) | Skills与现网 `_skill_keys_in_board_item` / `skill_id` 一致GitHub`owner/repo` | post-render persist | `board_select` / pad |
| `data.featured_pick_key` | 滚动 30 天 | skill id 或 `owner/repo` | `featured_resolve` 成功后 | 月去重 |
| `data.narrative_axis` | 滚动 3 天 | 枚举字符串 | 代码 `pick_narrative_axis` | Step 1 约束 / 校验 |
| `data.theme_line` / trends opening | 近 7 日供 prompt | 原文 | editorial 落盘 | Step 1 软禁(不硬匹配) |
| `CACHE_DIR/pushed-news-links.json` | `DAILY_NEWS_DEDUP_DAYS` | 规范化 URL | 推送成功后 | news filter |
不另建平行 CACHE「board-history.json」避免双源漂移。冷启动缺文件 = 空集合。
### 1.4 数据流挂点
| 逻辑 | 挂点 |
|------|------|
| `movement_baseline` | 现网raw 榜入库时(不变) |
| `board_select` | 渲染前;输出写入供 Agent/`llm_input` 与 wecom 共用的最终列表字段 |
| delta pad | **调用同一周去重历史**`wecom_shown_keys`),不再独立解释 `movement_baseline` 为展示史 |
| `featured_resolve` | **先于** why 检索;池 A = 本 run `board_select`delta 则为本 run 最终展示列表) |
| 新闻 | 所有 prepare 路径关 backfillresearch SKILL 改文案规则;后处理剥「放宽」 |
| `narrative_axis` | 代码先选轴再注入 Step 1LLM 不得另选冲突轴 |
| `wecom_shown_keys` 写回 | `replace_wecom_*` / `build_wecom_report` 之后,与最终正文列表一致 |
---
## Section 2 — 各榜选条 + 今日首推改推
### 2.1 `board_select`(五榜共用)
适用:`skills_trending` / `skills_hot` / `github_trending` / `github_emerging` / `github_topic`
```
输入:当日 raw 池pool ≥ DAILY_BOARD_POOL_SIZE
历史:近 DAILY_BOARD_DEDUP_DAYS 的 wecom_shown_keys[board]
输出:至多 N 条N = 现有 wecom Top 配置)
1. 现有整理Skillssource 合并GitHubrepo key
2. 滤掉近 7 天该榜 wecom_shown_keys
3. 按原排名取前 N
4. 不足 → 继续扫深池,仍排除周历史,直到满 N 或池空
5. 池空仍不足 → 短榜;日志 board_short:{board}:{n};不回填周内已展示条目
```
分榜独立历史Trending 出过的 skillHot 仍可出。
Post-render将**实际写入企微的** keys 写入当日 `wecom_shown_keys`测试断言history ⊆ / == 渲染列表,**≠** `movement_baseline`)。
### 2.2 `featured_resolve`
**触发**:本 run 拟用首推身份与**前一天** `featured_pick_key`(或等价 data 字段)相同。
身份函数skill → `skill_id`github → `owner/repo`
`DAILY_FEATURED_PICK` 与自动首推;本期不豁免。无昨日文件 → 不改推。
**顺序(硬)**:定候选 → 再 `research`/`why_today`(禁止先写旧条目 why 再改人却不重写)。
**候选**
1. **池 A**:本 run **最终会展示**的 Skills + GitHub 榜条目(与 `wecom_shown_keys` 同源结构)
2. **池 B**raw 深池中尚未进入本 run 展示者
**过滤**:近 30 天 `featured_pick_key`;排除冲突项自身。
**抽取**`hash(date_str + "featured")` 可复现;测试可注入 RNG。先 A 后 B仍空 → 保留原首推 + `featured_fallback_exhausted`
**落盘**`data.featured_pick_key`
---
## Section 3 — 开场主题 + AI 时讯
### 3.1 开场主题
| 机制 | 强度 | 规则 |
|------|------|------|
| `narrative_axis` | **硬** | 代码 `pick_narrative_axis(used_last_N)` 从剩余枚举选取;注入 promptLLM 输出须等于该轴;冲突则重试 1 次,再失败则**强制覆写为代码所选轴**再落盘(保证成功标准可测) |
| theme/opening 软禁 | **软** | prompt 附近 7 天 `theme_line`/opening 摘要;禁止复述;**无** n-gram 硬匹配;**不**列入硬成功标准 |
**叙事轴枚举**
`政策监管` · `模型发布` · `工具链/Agent` · `芯片算力` · `开源生态` · `应用落地` · `安全/诉讼`
`opening` 首句证据须来自当日数据;首推改推后须跟新首推或当日主轴新闻。
### 3.2 AI 时讯:取消「放宽窗口」
目标条数 = 现网配置之和(如 `DAILY_WECOM_AI_NEWS` + tech/CN 等文档不写死「15」。
1. **所有 prepare 路径**关闭「不够塞回已推/旧条」(`DAILY_NEWS_BACKFILL=0` 默认);`pushed-news-links` 过滤保留。
2. 不够 → 深度检索补新闻https link、未 pushed、可核实**补入年龄上限** = `DAILY_AI_NEWS_HOURS`(与主窗一致),禁止借 research 变相放宽到任意旧闻。
3. 改 research SKILL删除「放宽至 48h 并注明」;后处理剥 `放宽窗口`/`放宽至` 前缀或丢弃。
4. 仍不足 → 短列表 + `news_short:{n}`
中文化:`daily/localize.py`(不变)。
---
## Section 4 — 配置、错误处理、测试
### 4.1 环境变量
| 变量 | 默认 | 含义 |
|------|------|------|
| `DAILY_BOARD_DEDUP_DAYS` | `7` | 读 `wecom_shown_keys` 的滚动天数 |
| `DAILY_BOARD_POOL_SIZE` | ≥50 / 与现有 skill pool 对齐 | 深池扫描深度 |
| `DAILY_FEATURED_DEDUP_DAYS` | `30` | 今日首推月去重 |
| `DAILY_THEME_BAN_DAYS` | `7` | 软禁:注入 prompt 的 theme 天数 |
| `DAILY_NARRATIVE_AXIS_DAYS` | `3` | 叙事轴互斥窗 |
| `DAILY_NEWS_BACKFILL` | `0` | `0`=禁止旧闻凑数 |
| `DAILY_NEWS_DEDUP_DAYS` | 已有 `7` | pushed-links |
`DAILY_DELTA_PAD_LOOKBACK_DAYS` 应与 `DAILY_BOARD_DEDUP_DAYS` 对齐,且 **pad 与 board_select 共用 `wecom_shown_keys`**(窗口对齐不够,数据源必须同一)。
### 4.2 错误与降级
| 情况 | 行为 |
|------|------|
| 无 `wecom_shown_keys` 历史 | 空集合,正常满榜 |
| 周去重后深池不足 | 短榜 + `board_short` |
| 首推冲突且 A/B 空 | 保留原首推 + `featured_fallback_exhausted` |
| LLM 轴与代码轴冲突 | 覆写为代码轴 |
| 深检索仍不足时讯 | 短列表;禁止 backfill |
| history 读写失败 | 当次按空历史 + error 日志 |
### 4.3 测试pytest
1. `board_select`:假 `wecom_shown_keys` + 深池 → 无周交集;深池补满;不足短榜
2. **回归钉死**:写回后 `wecom_shown_keys` ≠ 用 `movement_baseline` 推导的集合(构造 raw Top 与展示 Top 故意不同)
3. deltapad 不引入近 7 日 `wecom_shown_keys` 内 key
4. `featured_resolve`:先定人再 whyA 优先 B月未见可注入 RNG
5. news`BACKFILL=0`;剥「放宽*」research 补入不超 hours 窗
6. `pick_narrative_axis`:近 3 天互斥;落盘轴 == 代码轴
7. 既有 delta / pushed_links / wecom 回归不挂
### 4.4 成功标准(硬)
- 连续两天:**首推 key 不同**(除非 `featured_fallback_exhausted`
- 同一榜近 7 日 `wecom_shown_keys`**无重复 key**(池足够时)
- 时讯:无「放宽*」标注;无 backfill 已推 link
- 近 3 天 `narrative_axis`**两两不同**(代码保证)
软标准不闸门opening 读感不像连续复印。
---
## Implementation Sketch非计划明细
1. `data.json` 增加 `wecom_shown_keys`;改 `load_recent_*` 数据源
2. `board_select` + 让 delta pad 共用
3. post-render persist shown keys
4. `featured_resolve` 时序修正
5. news backfill off + SKILL + 剥前缀
6. `pick_narrative_axis` + prompt 注入
7. 测试如上
正式任务拆解 → `writing-plans`
## Office-hours Review Notes
- 对抗审阅质量约 4/10 → 本修订处理三大硬伤(存储拆分、列表主人、真相表)。
- 未纳入本期(原选项 B首推质量加权、关键短语硬匹配。
- 已知残留source 级「同类」周内可再现。
## Spec Self-Review
- [x] `movement_baseline``wecom_shown_keys` 职责分离写死
- [x] 单一列表主人 + full/delta 交互说明
- [x] 历史真相表无「与/或」双源
- [x] 轴硬 / 短语软;成功标准不含无法验证的短语匹配
- [x] 周不足=深池、首推=A→B、新闻禁放宽 与访谈一致

View File

@@ -1,3 +0,0 @@
[pytest]
testpaths = tests
pythonpath = .

View File

@@ -1,4 +0,0 @@
python-dotenv>=1.0.0
httpx>=0.27.0
certifi>=2024.0.0
pytest>=8.0

View File

@@ -1,6 +1,5 @@
python-dotenv>=1.0.0 python-dotenv>=1.0.0
httpx>=0.27.0 httpx>=0.27.0
certifi>=2024.0.0 certifi>=2024.0.0
PyYAML>=6.0.0
# 英文描述转中文(使用 CURSOR_API_KEY 时需安装) # 英文描述转中文(使用 CURSOR_API_KEY 时需安装)
cursor-sdk>=0.1.0 cursor-sdk>=0.1.0

View File

@@ -3,21 +3,16 @@
# .\run-daily.ps1 # .\run-daily.ps1
# .\run-daily.ps1 -SkipPush # .\run-daily.ps1 -SkipPush
# .\run-daily.ps1 -SkipGenerate # .\run-daily.ps1 -SkipGenerate
# .\run-daily.ps1 -Force # bypass duplicate-run lock
param( param(
[switch]$SkipPush, [switch]$SkipPush,
[switch]$SkipGenerate, [switch]$SkipGenerate
[switch]$Force
) )
$ErrorActionPreference = "Stop" $ErrorActionPreference = "Stop"
$Root = Split-Path -Parent $MyInvocation.MyCommand.Path $Root = Split-Path -Parent $MyInvocation.MyCommand.Path
Set-Location $Root Set-Location $Root
$LockFile = Join-Path $Root ".cache\run-daily.lock"
$LockMaxMinutes = 30
function Import-DotEnvFile { function Import-DotEnvFile {
param([string]$Path) param([string]$Path)
if (-not (Test-Path $Path)) { return } if (-not (Test-Path $Path)) { return }
@@ -34,61 +29,30 @@ function Import-DotEnvFile {
} }
} }
function Test-RunDailyLock { Import-DotEnvFile (Join-Path $Root ".env")
if (-not (Test-Path $LockFile)) { return $false } Import-DotEnvFile (Join-Path $Root ".env.local")
$age = (Get-Date) - (Get-Item $LockFile).LastWriteTime
return $age.TotalMinutes -lt $LockMaxMinutes
}
function Set-RunDailyLock { $python = "python"
$dir = Split-Path $LockFile -Parent $date = Get-Date -Format "yyyy-MM-dd"
if (-not (Test-Path $dir)) { $reportWecom = Join-Path $Root "output\$date.wecom.md"
New-Item -ItemType Directory -Path $dir -Force | Out-Null
}
Set-Content -Path $LockFile -Value (Get-Date -Format "o") -Encoding UTF8
}
function Clear-RunDailyLock { if (-not $SkipGenerate) {
if (Test-Path $LockFile) { Write-Host "Generating daily report: $date"
Remove-Item $LockFile -Force -ErrorAction SilentlyContinue & $python -m daily generate
if ($LASTEXITCODE -ne 0) {
throw "daily generate failed with exit code $LASTEXITCODE"
} }
} }
if (-not $Force -and (Test-RunDailyLock)) { if (-not $SkipPush) {
Write-Host "Skip: run-daily already ran within ${LockMaxMinutes} minutes (lock: $LockFile). Use -Force to override." if (-not (Test-Path $reportWecom)) {
exit 0 throw "Report not found: $reportWecom"
}
Set-RunDailyLock
try {
Import-DotEnvFile (Join-Path $Root ".env")
Import-DotEnvFile (Join-Path $Root ".env.local")
$python = "python"
$date = Get-Date -Format "yyyy-MM-dd"
$reportWecom = Join-Path $Root "output\$date.wecom.md"
if (-not $SkipGenerate) {
Write-Host "Generating daily report: $date"
& $python -m daily generate
if ($LASTEXITCODE -ne 0) {
throw "daily generate failed with exit code $LASTEXITCODE"
}
} }
Write-Host "Pushing to WeCom webhook..."
if (-not $SkipPush) { & $python -m daily push $reportWecom
if (-not (Test-Path $reportWecom)) { if ($LASTEXITCODE -ne 0) {
throw "Report not found: $reportWecom" throw "daily push failed with exit code $LASTEXITCODE"
}
Write-Host "Pushing to WeCom webhook..."
& $python -m daily push $reportWecom
if ($LASTEXITCODE -ne 0) {
throw "daily push failed with exit code $LASTEXITCODE"
}
} }
}
Write-Host "Done: $date" Write-Host "Done: $date"
}
finally {
Clear-RunDailyLock
}

View File

@@ -1,106 +0,0 @@
#!/usr/bin/env bash
# Daily report: generate + push to WeCom webhook
# Usage:
# ./run-daily.sh
# ./run-daily.sh --skip-push
# ./run-daily.sh --skip-generate
# ./run-daily.sh --force
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$ROOT"
LOCK_FILE="$ROOT/.cache/run-daily.lock"
LOCK_MAX_MINUTES=30
SKIP_PUSH=0
SKIP_GENERATE=0
FORCE=0
while [[ $# -gt 0 ]]; do
case "$1" in
--skip-push) SKIP_PUSH=1 ;;
--skip-generate) SKIP_GENERATE=1 ;;
--force) FORCE=1 ;;
-h|--help)
echo "Usage: $0 [--skip-push] [--skip-generate] [--force]"
exit 0
;;
*)
echo "Unknown option: $1" >&2
exit 1
;;
esac
shift
done
load_dotenv() {
local file="$1"
[[ -f "$file" ]] || return 0
while IFS= read -r line || [[ -n "$line" ]]; do
[[ "$line" =~ ^[[:space:]]*# ]] && continue
[[ "$line" != *"="* ]] && continue
local name="${line%%=*}"
local value="${line#*=}"
name="${name#"${name%%[![:space:]]*}"}"
name="${name%"${name##*[![:space:]]}"}"
value="${value#"${value%%[![:space:]]*}"}"
value="${value%"${value##*[![:space:]]}"}"
value="${value%\"}"
value="${value#\"}"
value="${value%\'}"
value="${value#\'}"
if [[ -n "$name" && -n "$value" ]]; then
export "$name=$value"
fi
done < "$file"
}
lock_active() {
[[ -f "$LOCK_FILE" ]] || return 1
local now lock_ts age_minutes
now="$(date +%s)"
lock_ts="$(date -r "$LOCK_FILE" +%s 2>/dev/null || stat -c %Y "$LOCK_FILE" 2>/dev/null || echo 0)"
age_minutes=$(( (now - lock_ts) / 60 ))
[[ "$age_minutes" -lt "$LOCK_MAX_MINUTES" ]]
}
set_lock() {
mkdir -p "$(dirname "$LOCK_FILE")"
date -Iseconds > "$LOCK_FILE"
}
clear_lock() {
rm -f "$LOCK_FILE"
}
if [[ "$FORCE" -eq 0 ]] && lock_active; then
echo "Skip: run-daily already ran within ${LOCK_MAX_MINUTES} minutes (lock: $LOCK_FILE). Use --force to override."
exit 0
fi
set_lock
trap clear_lock EXIT
load_dotenv "$ROOT/.env"
load_dotenv "$ROOT/.env.local"
PYTHON="${PYTHON:-python}"
DATE="$(TZ=Asia/Shanghai date +%Y-%m-%d)"
REPORT_WECOM="$ROOT/output/${DATE}.wecom.md"
if [[ "$SKIP_GENERATE" -eq 0 ]]; then
echo "Generating daily report: $DATE"
"$PYTHON" -m daily generate
fi
if [[ "$SKIP_PUSH" -eq 0 ]]; then
if [[ ! -f "$REPORT_WECOM" ]]; then
echo "Report not found: $REPORT_WECOM" >&2
exit 1
fi
echo "Pushing to WeCom webhook..."
"$PYTHON" -m daily push "$REPORT_WECOM"
fi
echo "Done: $DATE"

43
run-scheduler.ps1 Normal file
View File

@@ -0,0 +1,43 @@
# Start the daily report scheduler (generate @ 08:50, push @ 09:00 by default)
# Usage:
# .\run-scheduler.ps1
# .\run-scheduler.ps1 -DryRun
# .\run-scheduler.ps1 -Once
param(
[switch]$DryRun,
[switch]$Once
)
$ErrorActionPreference = "Stop"
$Root = Split-Path -Parent $MyInvocation.MyCommand.Path
Set-Location $Root
function Import-DotEnvFile {
param([string]$Path)
if (-not (Test-Path $Path)) { return }
Get-Content $Path -Encoding UTF8 | ForEach-Object {
if ($_ -match '^\s*#' -or $_ -notmatch '=') { return }
$pair = $_ -split '=', 2
if ($pair.Count -eq 2) {
$name = $pair[0].Trim()
$value = $pair[1].Trim().Trim('"').Trim("'")
if ($name -and $value) {
Set-Item -Path "Env:$name" -Value $value
}
}
}
}
Import-DotEnvFile (Join-Path $Root ".env")
Import-DotEnvFile (Join-Path $Root ".env.local")
$args = @("python", "-m", "daily", "schedule")
if ($DryRun) { $args += "--dry-run" }
if ($Once) { $args += "--once" }
Write-Host "Starting scheduler: $($args -join ' ')"
& $args[0] $args[1..($args.Length - 1)]
if ($LASTEXITCODE -ne 0) {
throw "scheduler failed with exit code $LASTEXITCODE"
}

View File

@@ -1,47 +0,0 @@
from pathlib import Path
import yaml
from daily.news.feeds_types import NewsCategory, NewsFeed
data = yaml.safe_load(Path("config/feeds.yaml").read_text(encoding="utf-8"))
def parse_feed(raw: dict) -> NewsFeed:
return NewsFeed(
name=raw["name"],
url=raw["url"],
slow=bool(raw.get("slow")),
ai_filter=bool(raw.get("ai_filter")),
)
def parse_cat(raw: dict) -> NewsCategory:
return NewsCategory(
id=raw["id"],
name=raw["name"],
icon=raw["icon"],
feeds=tuple(parse_feed(item) for item in raw["feeds"]),
)
intl = tuple(parse_cat(item) for item in data["intl"]["categories"])
cn = tuple(parse_cat(item) for item in data["cn"]["categories"])
keywords = tuple(data["cn"]["title_keywords"])
lines = [
'"""Built-in RSS defaults when config/feeds.yaml is missing or invalid."""',
"",
"from __future__ import annotations",
"",
"from daily.news.feeds_types import NewsCategory, NewsFeed",
"",
f"CN_AI_TITLE_KEYWORDS: tuple[str, ...] = {keywords!r}",
"",
f"CN_NEWS_CATEGORIES: tuple[NewsCategory, ...] = {cn!r}",
"",
f"NEWS_CATEGORIES: tuple[NewsCategory, ...] = {intl!r}",
"",
]
Path("daily/news/feeds_defaults.py").write_text("\n".join(lines), encoding="utf-8")
print("written")

View File

@@ -1,10 +0,0 @@
"""Repo-root shared modules (skills feed data, etc.)."""
from shared.skills_data import (
Board,
format_installs,
load_feed,
warm_feed_cache,
)
__all__ = ["Board", "format_installs", "load_feed", "warm_feed_cache"]

View File

@@ -1,109 +0,0 @@
"""skills.sh feed.json 拉取、缓存与榜单数据。"""
from __future__ import annotations
import json
import logging
import time
from pathlib import Path
from typing import Any, Literal
import certifi
import httpx
logger = logging.getLogger(__name__)
ROOT = Path(__file__).resolve().parent.parent
FEED_URLS = [
"https://cdn.jsdelivr.net/gh/NeverSight/skills.sh_feed@main/data/feed.json",
"https://raw.githubusercontent.com/NeverSight/skills.sh_feed/main/data/feed.json",
]
CACHE_TTL_SECONDS = 600
CACHE_DIR = ROOT / ".cache"
CACHE_FILE = CACHE_DIR / "skills-feed.json"
USER_AGENT = "Mozilla/5.0 (compatible; daily-robots/1.0; +https://skills.sh)"
_cache: dict[str, Any] = {"data": None, "fetched_at": 0.0}
Board = Literal["trending", "hot", "all"]
BOARD_FEED_KEYS = {
"trending": "topTrending",
"hot": "topHot",
"all": "topAllTime",
}
def _fetch_json(url: str) -> dict[str, Any]:
headers = {"User-Agent": USER_AGENT, "Accept": "application/json"}
with httpx.Client(
timeout=httpx.Timeout(20.0, connect=10.0),
verify=certifi.where(),
follow_redirects=True,
) as client:
resp = client.get(url, headers=headers)
resp.raise_for_status()
return resp.json()
def _load_disk_cache() -> dict[str, Any] | None:
if not CACHE_FILE.exists():
return None
try:
return json.loads(CACHE_FILE.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
logger.warning("读取 skills feed 本地缓存失败: %s", exc)
return None
def _save_disk_cache(data: dict[str, Any]) -> None:
CACHE_DIR.mkdir(parents=True, exist_ok=True)
CACHE_FILE.write_text(json.dumps(data, ensure_ascii=False), encoding="utf-8")
def load_feed(force: bool = False) -> dict[str, Any]:
now = time.time()
if not force and _cache["data"] and now - _cache["fetched_at"] < CACHE_TTL_SECONDS:
return _cache["data"]
errors: list[str] = []
for url in FEED_URLS:
for attempt in range(3):
try:
data = _fetch_json(url)
_cache["data"] = data
_cache["fetched_at"] = now
_save_disk_cache(data)
logger.info("skills 数据已更新: %s", url)
return data
except Exception as exc:
msg = f"{url} (#{attempt + 1}): {exc}"
errors.append(msg)
logger.debug("拉取失败 %s", msg)
time.sleep(0.5 * (attempt + 1))
stale = _load_disk_cache()
if stale:
logger.warning("网络不可用,回退到本地 skills feed 缓存")
_cache["data"] = stale
_cache["fetched_at"] = now
return stale
raise RuntimeError(f"无法获取 skills 数据。最近错误: {errors[-1]}")
def warm_feed_cache() -> None:
"""启动时预加载,避免首条消息才触发网络请求。"""
load_feed(force=True)
def format_installs(n: int | float) -> str:
if n >= 1_000_000:
return f"{n / 1_000_000:.1f}M"
if n >= 1_000:
return f"{n / 1_000:.1f}K"
return str(int(n))
def board_items(feed: dict[str, Any], board: Board) -> list[dict[str, Any]]:
return feed.get(BOARD_FEED_KEYS[board], [])

View File

@@ -1,10 +1,11 @@
# 早报 Agent 工作流 # 早报 Agent 工作流
你是 **Skills / GitHub / AI 时讯早报** 的主编 Agent。Python 已完成数据抓取;**两步** 产出可读性强的企微早报。 你是 **Skills / GitHub / AI 时讯早报** 的主编 Agent。Python 已完成数据抓取;若配置了 `DAILY_FEATURED_PICK`Python 会先完成 **Step 0 首推检索** 并注入 `featured_pick`。你再**两步** 产出可读性强的企微早报。
## 工作流 ## 工作流
``` ```
Step 0 首推检索Python + daily-featured-pick输出 featured_pick
Step 1 读话题 → 识别热门趋势(输出 trends JSON Step 1 读话题 → 识别热门趋势(输出 trends JSON
Step 2 基于趋势 + 原始数据 → 写企微 Markdown 早报 Step 2 基于趋势 + 原始数据 → 写企微 Markdown 早报
``` ```
@@ -27,21 +28,31 @@ Step 2 基于趋势 + 原始数据 → 写企微 Markdown 早报
| `github_emerging` | GitHub **新兴** 当前 Top N | | `github_emerging` | GitHub **新兴** 当前 Top N |
| `github_topic.repos` | GitHub **Topic** 当前 Top N | | `github_topic.repos` | GitHub **Topic** 当前 Top N |
| `github_topic.topic` | Topic 名称,用于区块标题(如 `llm` | | `github_topic.topic` | Topic 名称,用于区块标题(如 `llm` |
| `movement.*_moves` | 较昨日新增(**仅**用于 opening / signals**不**用于列表区块 | | `movement.*_moves` | 较昨日新增(**仅**用于 opening / signals`effective_wecom_mode=delta` 时列表由 Python 插入 |
| `movement.*_summary` | 新增摘要(可选写入 signals | | `movement.*_summary` | 新增摘要(可选写入 signals |
| `ai_news` | 国际 AI 时讯 Top N | | `ai_news` | 国际 AI 时讯 Top NRSS 模式) |
| `cn_ai_news` | 国内 AI 时讯 Top N | | `cn_ai_news` | 国内 AI 时讯 Top NRSS 模式) |
| `ai_news_mode` | `research` 时仅 `ai_news` 有 10 条合并精选,`cn_ai_news` 为空 |
| `featured_pick` | **可选**编辑指定今日首推Step 0 产出;含 command / why_today / evidence |
**禁止**使用已合并的 `skills_moves` / `github_moves` 自行扩写;**禁止**排名变化、安装涨跌。 **禁止**使用已合并的 `skills_moves` / `github_moves` 自行扩写;**禁止**排名变化、安装涨跌。
Step 1 的 `signals``top_picks` **优先引用 Top 榜榜首/前列条目、movement 新增与 AI 时讯** Step 1 的 `signals``top_picks` **优先引用 Top 榜榜首/前列条目、movement 新增与 AI 时讯**
### 当输入含 `featured_pick` 时Step 1
- `top_picks.skill` **必须**使用 `featured_pick``id`/`title`/`command`/`url` 来自 featured`why` 来自 `featured_pick.why_today`
- `opening` 首句优先引用 `featured_pick.evidence` 中的数字或条目名
- `signals` 至少 1 条与 featured 相关
- featured 不在 Top 榜时仍可作为首推,`why` 须来自 `featured_pick.evidence`,禁止编造榜单排名
**只输出 JSON** **只输出 JSON**
```json ```json
{ {
"headline": "816字焦点标题", "headline": "816字焦点标题",
"opening": "23句中文导语首句必须是具体证据榜首 skill+安装量 / 头条新闻 / GitHub #1再解释为什么值得看", "opening": "23句中文导语首句必须是具体证据榜首 skill+安装量 / 头条新闻 / GitHub #1再解释为什么值得看",
"narrative_axis": "必须等于输入 required_narrative_axis政策监管|模型发布|工具链/Agent|芯片算力|开源生态|应用落地|安全/诉讼)",
"themes": [ "themes": [
{ {
"title": "主题名", "title": "主题名",
@@ -50,7 +61,7 @@ Step 1 的 `signals` 与 `top_picks` **优先引用 Top 榜榜首/前列条目
} }
], ],
"top_picks": { "top_picks": {
"skill": { "id": "owner/repo/skill", "title": "...", "why": "中文,为什么今天首推" }, "skill": { "id": "owner/repo/skill", "title": "...", "command": "npx skills add ...", "why": "中文,为什么今天首推" },
"github": { "repo": "owner/repo", "why": "中文" }, "github": { "repo": "owner/repo", "why": "中文" },
"news": { "link": "完整URL", "title_zh": "中文标题", "why": "中文一句话" } "news": { "link": "完整URL", "title_zh": "中文标题", "why": "中文一句话" }
}, },
@@ -64,16 +75,17 @@ Step 1 的 `signals` 与 `top_picks` **优先引用 Top 榜榜首/前列条目
要求: 要求:
- 所有结论必须能在输入 JSON 中找到依据,禁止编造 - 所有结论必须能在输入 JSON 中找到依据,禁止编造
- `narrative_axis` **必填**,且必须等于输入中的 `required_narrative_axis`(代码已选定;勿自选其它轴)
- `opening` 遵循 **article-writing Newsletter** 规则:首句用数字/条目名/新闻标题开头,不用「今天有三条线」「值得关注」等空框架 - `opening` 遵循 **article-writing Newsletter** 规则:首句用数字/条目名/新闻标题开头,不用「今天有三条线」「值得关注」等空框架
- `signals` 35 条,每条单行,可含 emoji 前缀;与 `opening` 不重复同一句信息 - `signals` 35 条,每条单行,可含 emoji 前缀;与 `opening` 不重复同一句信息
- `top_picks.why` 用「事实/数字 + 一句判断」,不用空泛形容词 - `top_picks.why` 用「事实/数字 + 一句判断」,不用空泛形容词
- `top_picks` 必须引用输入中真实存在的 id/repo/link - `top_picks` 必须引用输入中真实存在的 id/repo/link;有 `featured_pick` 时 skill 首推以 featured 为准
--- ---
## Step 2撰写企微早报 ## Step 2撰写企微早报
你会收到 **原始数据 JSON** + **Step 1 的 trends JSON** 你会收到 **原始数据 JSON**(含可选 `featured_pick`+ **Step 1 的 trends JSON**
**只输出企微 Markdown 正文**(不要代码块包裹,不要解释)。 **只输出企微 Markdown 正文**(不要代码块包裹,不要解释)。
@@ -93,21 +105,33 @@ Step 1 的 `signals` 与 `top_picks` **优先引用 Top 榜榜首/前列条目
> {signal 3} > {signal 3}
📦 **今日首推** 📦 **今日首推**
`npx skills add {source}/{skill}` [{featured_pick.title 或 repo 或 skill 名}]({featured_pick.url 或 command 或 skills.sh 链接})
> {top_picks.skill.why} > {featured_pick.why_today 或 top_picks.skill.why}
### 当输入含 `featured_pick` 时Step 2
- **今日首推**链接行用 Markdown `[标题](URL)`,与新闻条目同格式;**不要**用反引号裸 URL 或裸 `npx` 命令
- GitHub 首推:标题用 `owner/repo`(如 `[garrytan/gstack](https://github.com/...)`
- Skill 首推:标题用 skill 名,链接用 `featured_pick.url``https://skills.sh/{id}`
- 说明用 `featured_pick.why_today`**不要**改回 Trending 榜首
- 读者可见文案opening / 今日首推说明 / why**不得**写「编辑指定首推」等元信息,只陈述项目事实与判断
🌍 **国际 AI · 精选 10** 🌍 **国际 AI · 精选 10**
1. [{title_zh}]({link}) — {why 或摘要} 1. [{title_zh}]({link}) — {why 或摘要}
2. ...**必须 10 条**,来自 `ai_news`,按重要性排序) 2. ...**必须 10 条**,来自 `ai_news`,按重要性排序)
🇨🇳 **国内 AI · 精选 8** 🇨🇳 **国内 AI · 精选 10**
1. [{title}]({link}) — {why 或摘要} 1. [{title}]({link}) — {why 或摘要}
2. ...**必须 8 条**,来自 `cn_ai_news`,按重要性排序;标题已是中文,可微调润色 2. ...**必须 10 条**,来自 `cn_ai_news`RSS 模式
<!-- **不要写** Skills Trending / Skills Hot 区块Python 会在推送前按 source 合并后自动插入 --> 📰 **AI 时讯精选 · 15**(当 `ai_news_mode=research`
1. [{title}]({link}) — {why 或摘要}
2. ...**必须 15 条**:前 10 条综合精选 + 后 5 条偏工程技术,来自 `ai_news``tech_ai_news`**不要**再写 🌍/🇨🇳/🔧 分块)
<!-- delta 模式effective_wecom_mode=delta不要写任何 Skills / GitHub 榜单区块Python 会插入变化列表 -->
🐙 **GitHub Trending Top {N}** 🐙 **GitHub Trending Top {N}**
<!-- 只列 data.github_trending --> <!-- full 模式:只列 data.github_trending -->
1. [{repo}]({url}) · {lang} · ⭐{stars} — {中文一句话} 1. [{repo}]({url}) · {lang} · ⭐{stars} — {中文一句话}
🌱 **GitHub 新兴 Top {N}** 🌱 **GitHub 新兴 Top {N}**
@@ -121,14 +145,16 @@ Step 1 的 `signals` 与 `top_picks` **优先引用 Top 榜榜首/前列条目
### 榜单选取规则top_n ### 榜单选取规则top_n
1. **五个 GitHub 区块分开写**GitHub Trending / 新兴 / Topic**禁止合并** 1. **五个 GitHub 区块分开写**(仅 `effective_wecom_mode=full`GitHub Trending / 新兴 / Topic**禁止合并**
2. **Skills Trending / Hot 由 Python 自动插入**Agent 不要写这两段 2. **Skills Trending / Hot 由 Python 自动插入**Agent 不要写这两段
3. **禁止**改用 `movement.*_moves` 作为列表来源movement 用于 opening / signals 描述「今日新增」 3. `data.effective_wecom_mode``delta` 时:**禁止写** Skills Trending / Hot / GitHub 列表区块Python 插入变化列表)movement 用于 opening / signals`full` 模式保持原 GitHub 列表规则
4. **禁止**在条目后写 `(新入 … #n` 类括号标注 4. **禁止**在条目后写 `(新入 … #n` 类括号标注
5. **即使某榜较昨日无新增,仍须完整列出 Top 榜条目** 5. `effective_wecom_mode=full` 时:即使某榜较昨日无新增,仍须完整列出 Top 榜条目
6. **国际 AI 必须 10 条**(来自 `ai_news` 6. **国际 AI 必须 10 条**(来自 `ai_news`
7. **国内 AI 必须 8 条**(来自 `cn_ai_news`;无数据时写「暂无可用条目」) 7. **国内 AI 必须 10 条**(来自 `cn_ai_news`;无数据时写「暂无可用条目」)
8. 禁止排名变化、安装涨跌、连霸描述 8. **`ai_news_mode=research` 时**:只写 **📰 AI 时讯精选 15 条**`ai_news` 10 条 + `tech_ai_news` 5 条合并展示),不写 🌍/🇨🇳/🔧 分块Python 会用调研结果覆盖该区块
9. 禁止排名变化、安装涨跌、连霸描述
10. 榜全稳movement 各 `*_stable` 为 truesignals 聚焦新闻与首推,不编造榜单变化
```markdown ```markdown
📈 **Skills Trending Top 10** 📈 **Skills Trending Top 10**
@@ -161,6 +187,7 @@ Step 1 的 `signals` 与 `top_picks` **优先引用 Top 榜榜首/前列条目
**禁止Banned Patterns** **禁止Banned Patterns**
- 「据悉」「值得关注」「快速演进」「In today's rapidly evolving landscape」 - 「据悉」「值得关注」「快速演进」「In today's rapidly evolving landscape」
- 「编辑指定首推」「编辑今日首推」等内部流程用语(读者不应感知编辑配置)
- 「今天有三条线叠在一起」这类无证据的空框架开场 - 「今天有三条线叠在一起」这类无证据的空框架开场
- 无证据的「为什么这很重要」「 here's why this matters」 - 无证据的「为什么这很重要」「 here's why this matters」
- 结尾硬塞互动问句(如「你怎么看?」「值得花十分钟扫一眼」) - 结尾硬塞互动问句(如「你怎么看?」「值得花十分钟扫一眼」)

View File

@@ -0,0 +1,58 @@
# AI 时讯 Deep Research早报专用
你是 **AI 时讯调研员**。使用 **WebSearch** 与网页抓取工具,收集近 N 小时全球 AI 新闻(不区分国内/国外),输出供企微早报使用的结构化 JSON。
## 工作流
1. 将任务拆成 35 个子问题(模型发布、监管政策、大厂动态、芯片算力、研究突破等)
2. 每个子问题用 WebSearch 检索 23 组关键词(中英文混合)
3. 交叉验证:优先权威媒体 / 官方博客 / 学术来源
4. 精选最多 **10 条**最重要、可核实的新闻(`items`);窗口内不足则少返回,勿凑数
5. 另精选最多 **5 条**工程技术向新闻(`tech_items`):模型/框架发布、开源、芯片算力、开发者工具、推理与工程实践;不得与 `items` 重复 link不足则少返回
6. **只输出 JSON**,不要 Markdown 报告,不要代码块
## 质量规则
1. 每条必须有可访问的 `link`https://
2. 禁止编造未在搜索结果中出现的事实
3. 优先近 N 小时内的新闻;若不足目标条数,**少返回**即可,禁止放宽至 48 小时凑数,禁止在 `desc_short` 标注「放宽窗口」
4. `desc_short` 用中文一句话摘要≤72 字)
5. `title` 保留原文标题;中文源可用中文标题
6. `source_name` 为媒体/站点简称(如 TechCrunch、量子位、OpenAI Blog
## 输出格式(严格 JSON
```json
{
"items": [
{
"title": "Apple sues OpenAI over trade secret theft",
"link": "https://techcrunch.com/...",
"source_name": "TechCrunch",
"desc_short": "苹果起诉 OpenAI 涉嫌窃取硬件商业机密",
"published_fmt": "07-11 05:00"
}
],
"tech_items": [
{
"title": "Meta Iris AI chip enters production",
"link": "https://example.com/...",
"source_name": "TechCrunch",
"desc_short": "Meta 自研 Iris 芯片 9 月量产",
"published_fmt": ""
}
],
"methodology": "检索 6 组 query分析 12 源,子问题:诉讼、模型安全、监管"
}
```
- `items` 数组长度 **必须等于** 请求的 limit默认 10
- `tech_items` 数组长度 **必须等于** 请求的 tech limit默认 5聚焦工程技术可与 `items` 主题重叠但 link 不得重复
- `published_fmt` 格式 `MM-DD HH:MM`UTC+8无法确定则留空字符串
- 不要输出 `items` 以外的长文;`methodology` 可选,一行即可
## 禁止
- 不要输出 ```json 代码块包裹(直接输出 JSON 对象)
- 不要输出 Executive Summary / Key Takeaways 等报告章节
- 不要使用本项目 RSS 或本地文档作为来源

View File

@@ -0,0 +1,87 @@
# 早报今日首推检索
你是 **Skills / GitHub / AI 时讯早报** 的编辑研究员。Python 已完成榜单抓取;你负责为 **编辑指定的今日首推** 收集可核实信息,供后续趋势分析与写稿使用。
## 场景
- 触发:环境变量 `DAILY_FEATURED_PICK` 有值(如 `gstack``gstack|https://github.com/you/gstack`
- 输出:严格 JSON写入 `output/YYYY-MM-DD.featured.json`
-**不** 写整篇早报、 **不** 改榜单顺序、 **不** 推送
## 输入
```json
{
"query": "gstack",
"url_hint": "https://github.com/you/gstack",
"cwd": "D:\\path\\to\\workspace",
"data_matches": {
"skills": [],
"github": []
}
}
```
| 字段 | 含义 |
|------|------|
| `query` | 编辑指定的关键词skill 名 / repo 名片段) |
| `url_hint` | 可选,项目主页或仓库 URL |
| `cwd` | Cursor 工作目录,可在此检索 README / SKILL.md |
| `data_matches` | Python 已在今日 Top 榜中预匹配的条目(**优先使用其数字** |
## 检索顺序
1. **读 `data_matches`**:若 skills/github 有匹配, installs / star / repo / link **必须来自此处**,不得改写
2. **读 `cwd` 本地仓库**:搜索 README、SKILL.md、package.json 描述,提炼「做什么 + 技术栈」
3. **用 `url_hint`**:作为项目主页;无本地文件时可仅基于 URL 与 query 写 summary须标注 evidence 来源)
4. **禁止编造**:未在 data_matches / 本地文件 / url_hint 出现的数字、功能、版本一律不写
## 输出
**只输出一个 JSON 对象**,不要 markdown 围栏,不要解释。
```json
{
"title": "gstack",
"type": "skill|github|other",
"command": "npx skills add owner/repo/skill",
"url": "https://...",
"summary": "23 句中文:做什么 + 技术栈/场景",
"why_today": "中文,为什么今天主推(事实 + 一句判断)",
"evidence": ["Skills Trending 匹配 · remotion-render · 22.3K", "README: Agent 工作流 CLI"],
"tags": ["agent", "workflow"]
}
```
### 字段要求
| 字段 | 要求 |
|------|------|
| `title` | 展示名,通常与 query 或匹配条目 title/repo 短名一致 |
| `type` | `skill` = Skills 条目;`github` = 仓库;`other` = 仅关键词/URL |
| `command` | Skill`npx skills add {source}/{title}`GitHub仓库 URLotherurl_hint 或 query |
| `url` | 可点击链接,来自 data_matches.link / repo url / url_hint |
| `summary` | 3680 字中文,动词开头,说清用途 |
| `why_today` | 4080 字,「事实/数字 + 判断」,不用空泛形容词;**不得**出现「编辑指定」「编辑首推」等内部流程用语 |
| `evidence` | 24 条短字符串,标明信息来源 |
| `tags` | 04 个英文或中文关键词 |
### type 与 command 示例
- Skill 匹配:`type=skill``command=npx skills add vercel-labs/skills/find-skills`
- GitHub 匹配:`type=github``command=https://github.com/openclaw/openclaw`
- 仅关键词:`type=other``command` 用 url_hint
## 写作原则
1. **事实优先**why_today 每条 claim 能在 evidence 或 data_matches 中找到
2. **数字必真**installs、star 与 data_matches 完全一致
3. **中文叙述**summary / why_today 全中文skill/repo 名保留英文
4. **克制**不写「值得关注」「game-changer」等空话
## 输出前自检
- [ ] 仅有 JSON无围栏、无前后说明
- [ ] data_matches 有数字时summary/why_today 已引用
- [ ] command / url 与 type 一致
- [ ] 未编造未检索到的事实

View File

@@ -1,50 +0,0 @@
from __future__ import annotations
import json
from datetime import datetime, timedelta, timezone
from pathlib import Path
import pytest
FIXTURES = Path(__file__).parent / "fixtures"
@pytest.fixture
def skills_feed() -> dict:
return json.loads((FIXTURES / "skills-feed.json").read_text(encoding="utf-8"))
@pytest.fixture
def fixed_cst():
return datetime(2026, 7, 3, 9, 30, tzinfo=timezone(timedelta(hours=8)))
@pytest.fixture
def isolated_output(tmp_path, monkeypatch):
import daily.config as config
import daily.report_data as report_data
out = tmp_path / "output"
cache = tmp_path / "cache"
logs = tmp_path / "logs"
out.mkdir()
cache.mkdir()
logs.mkdir()
monkeypatch.setattr(config, "OUTPUT_DIR", out)
monkeypatch.setattr(report_data, "OUTPUT_DIR", out)
monkeypatch.setattr(config, "CACHE_DIR", cache)
monkeypatch.setattr(config, "LOG_DIR", logs)
monkeypatch.setattr(config, "SNAPSHOT_FILE", cache / "last-report.json")
return out
@pytest.fixture
def offline_generate_env(monkeypatch):
monkeypatch.setenv("DAILY_AI_NEWS", "0")
monkeypatch.setenv("DAILY_CN_AI_NEWS", "0")
monkeypatch.setenv("DAILY_ZH_DESC", "0")
monkeypatch.setenv("SKILLS_BOARD_SOURCE", "feed")
monkeypatch.setenv("DAILY_REPORT_MODE", "classic")
monkeypatch.setenv("DAILY_CURSOR_EDITOR", "0")
monkeypatch.setenv("DAILY_TRENDING_LIMIT", "10")
monkeypatch.setenv("DAILY_HOT_LIMIT", "10")

View File

@@ -1,12 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0">
<channel>
<title>Sample AI Feed</title>
<item>
<title>Sample AI headline for smoke tests</title>
<link>https://example.com/ai-news/1</link>
<pubDate>Wed, 02 Jul 2026 08:00:00 GMT</pubDate>
<description>A minimal RSS item used by offline tests.</description>
</item>
</channel>
</rss>

View File

@@ -1,32 +0,0 @@
{
"updatedAt": "2026-07-02T12:00:00Z",
"topTrending": [
{
"id": "vercel-labs/skills/find-skills",
"title": "find-skills",
"source": "vercel-labs/skills",
"installs": 18000,
"link": "https://www.skills.sh/vercel-labs/skills/find-skills",
"description": "Help users discover and install Agent Skills."
},
{
"id": "halt-catch-fire/skills/remotion-render",
"title": "remotion-render",
"source": "halt-catch-fire/skills",
"installs": 21549,
"link": "https://www.skills.sh/halt-catch-fire/skills/remotion-render",
"description": "Render videos from React/Remotion component code."
}
],
"topHot": [
{
"id": "vercel-labs/skills/find-skills",
"title": "find-skills",
"source": "vercel-labs/skills",
"installs": 243,
"link": "https://www.skills.sh/vercel-labs/skills/find-skills",
"description": "Help users discover and install Agent Skills."
}
],
"topAllTime": []
}

View File

@@ -1,85 +0,0 @@
from __future__ import annotations
import json
def test_analyze_trends_parses_json(monkeypatch, isolated_output):
import daily.agent_workflow as agent
import daily.config as config
monkeypatch.setattr(config, "OUTPUT_DIR", isolated_output)
monkeypatch.setattr(agent, "OUTPUT_DIR", isolated_output)
monkeypatch.setattr(
agent,
"llm_chat",
lambda system, user: json.dumps(
{
"headline": "Skills 视频工具升温",
"opening": "今日 remotion 相关技能继续走强。",
"themes": [{"name": "视频", "summary": "Remotion 生态活跃"}],
"top_picks": [],
"signals": [],
},
ensure_ascii=False,
),
)
llm_input = {"date": "2026-07-03", "skills_trending": []}
trends = agent.analyze_trends(llm_input, date_str="2026-07-03")
assert trends is not None
assert trends["headline"] == "Skills 视频工具升温"
assert (isolated_output / "2026-07-03.trends.json").exists()
def test_write_wecom_report_extracts_markdown_block(monkeypatch):
import daily.agent_workflow as agent
monkeypatch.setattr(
agent,
"llm_chat",
lambda system, user: "```markdown\n📰 **早报 · 2026-07-03**\n\n正文\n```",
)
md = agent.write_wecom_report(
{"date": "2026-07-03"},
{"headline": "test", "opening": "open"},
date_str="2026-07-03",
time_str="09:30 (UTC+8)",
updated="2026-07-02",
)
assert md is not None
assert md.startswith("📰")
assert "正文" in md
def test_run_agent_workflow_retries_step2(monkeypatch):
import daily.agent_workflow as agent
calls = {"n": 0}
def fake_llm(system, user):
if '"trends"' in user:
calls["n"] += 1
if calls["n"] == 1:
raise RuntimeError("Bridge request timed out")
return "📰 **早报 · 2026-07-03**\n\n重试成功"
return json.dumps(
{"headline": "h", "opening": "o", "themes": [], "top_picks": [], "signals": []},
ensure_ascii=False,
)
monkeypatch.setattr(agent, "llm_chat", fake_llm)
md = agent.run_agent_workflow(
{"date": "2026-07-03"},
date_str="2026-07-03",
time_str="09:30 (UTC+8)",
updated="2026-07-02",
)
assert md is not None
assert "重试成功" in md
assert calls["n"] == 2

View File

@@ -0,0 +1,117 @@
"""AI 时讯 deep-research 解析与企微格式。"""
from __future__ import annotations
import unittest
from daily.format_wecom import _ai_news_lines, replace_wecom_news_sections
from daily.news.research import parse_research_response
class TestAiNewsResearchParse(unittest.TestCase):
def test_parse_items(self):
raw = """
{
"items": [
{
"title": "Apple sues OpenAI",
"link": "https://techcrunch.com/2026/07/10/apple/",
"source_name": "TechCrunch",
"desc_short": "苹果起诉 OpenAI 涉嫌窃取商业机密",
"published_fmt": "07-11 05:00"
}
]
}
"""
items, tech = parse_research_response(raw, limit=10)
self.assertEqual(len(items), 1)
self.assertEqual(items[0]["source_name"], "TechCrunch")
self.assertIn("苹果", items[0]["desc_short"])
self.assertEqual(tech, [])
def test_dedupe_links(self):
raw = """{"items": [
{"title": "A", "link": "https://example.com/a", "source_name": "Ex", "desc_short": ""},
{"title": "B", "link": "https://example.com/a", "source_name": "Ex", "desc_short": ""}
]}"""
items, _ = parse_research_response(raw, limit=10)
self.assertEqual(len(items), 1)
def test_parse_tech_items(self):
raw = """{"items": [
{"title": "A", "link": "https://example.com/a", "source_name": "Ex", "desc_short": ""}
], "tech_items": [
{"title": "B", "link": "https://example.com/b", "source_name": "Ex", "desc_short": ""}
]}"""
items, tech = parse_research_response(raw, limit=10, tech_limit=5)
self.assertEqual(len(items), 1)
self.assertEqual(len(tech), 1)
self.assertEqual(tech[0]["title"], "B")
class TestMergedWecomNews(unittest.TestCase):
def test_merged_block_format(self):
items = [
{
"title": "Apple sues OpenAI",
"link": "https://techcrunch.com/x",
"source_name": "TechCrunch",
"desc_short": "苹果起诉 OpenAI",
"published_fmt": "07-11 05:00",
}
]
lines = _ai_news_lines(items, merged=True)
self.assertIn("TechCrunch - Apple sues OpenAI", lines[0])
self.assertIn("— 苹果起诉 OpenAI", lines[0])
self.assertNotIn("07-11", lines[0])
def test_merged_with_tech_block(self):
md = """📰 **早报**
📈 **Skills Trending Top 1**
1. skill
"""
main = [
{"title": "Main", "link": "https://example.com/m", "source_name": "Src", "desc_short": "主条", "published_fmt": "07-11"}
]
tech = [
{"title": "Tech", "link": "https://example.com/t", "source_name": "Src2", "desc_short": "技术条", "published_fmt": "07-12"}
]
out = replace_wecom_news_sections(md, ai_news=main, tech_ai_news=tech, merged=True)
self.assertIn("📰 **AI 时讯精选 Top 2**", out)
self.assertNotIn("技术类时讯", out)
self.assertNotIn("🔧", out)
self.assertNotIn("07-11", out)
self.assertNotIn("07-12", out)
self.assertIn("2. [Src2 - Tech]", out)
def test_replace_merged_removes_split_blocks(self):
md = """📰 **早报**
🌍 **国际 AI 时讯 Top 1**
1. [old](https://example.com/old) · `X`
🇨🇳 **国内 AI 时讯 Top 1**
1. [old2](https://example.com/old2) · `Y`
📈 **Skills Trending Top 1**
1. skill
"""
items = [
{
"title": "New story",
"link": "https://example.com/new",
"source_name": "Fortune",
"desc_short": "新故事",
"published_fmt": "",
}
]
out = replace_wecom_news_sections(md, ai_news=items, merged=True)
self.assertIn("📰 **AI 时讯精选 Top 1**", out)
self.assertNotIn("国际 AI 时讯", out)
self.assertNotIn("国内 AI 时讯", out)
self.assertIn("📈 **Skills Trending Top 1**", out)
if __name__ == "__main__":
unittest.main()

140
tests/test_board_history.py Normal file
View File

@@ -0,0 +1,140 @@
# tests/test_board_history.py
from __future__ import annotations
import json
import os
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from daily.board_history import extract_shown_keys, load_recent_shown_keys, merge_wecom_shown_into_data
from daily.config import board_dedup_days, news_backfill_enabled
class ConfigDiversityTests(unittest.TestCase):
def test_board_dedup_days_default(self):
with patch.dict(os.environ, {}, clear=True):
self.assertEqual(board_dedup_days(), 7)
def test_news_backfill_default_off(self):
with patch.dict(os.environ, {}, clear=True):
self.assertFalse(news_backfill_enabled())
class ShownKeysTests(unittest.TestCase):
def test_extract_github_repo_keys(self):
items = [{"repo": "a/b"}, {"repo": "c/d"}]
self.assertEqual(extract_shown_keys("github_trending", items), ["a/b", "c/d"])
def test_extract_skill_keys_include_source(self):
items = [
{
"id": "open.feishu.cn/lark-drive",
"source": "open.feishu.cn",
"title": "lark-drive",
}
]
self.assertEqual(
extract_shown_keys("skills_trending", items),
["open.feishu.cn/lark-drive", "open.feishu.cn"],
)
def test_load_recent_reads_wecom_shown_not_baseline(self):
with tempfile.TemporaryDirectory() as tmp:
out = Path(tmp)
# 前日shown 只有 x/ybaseline raw 含 a/b —— 周去重只能看到 x/y
payload = {
"data": {
"date": "2026-07-13",
"movement_baseline": {
"github_trending": [{"repo": "a/b"}, {"repo": "x/y"}],
},
"wecom_shown_keys": {"github_trending": ["x/y"]},
}
}
(out / "2026-07-13.data.json").write_text(
json.dumps(payload, ensure_ascii=False), encoding="utf-8"
)
with patch("daily.board_history.OUTPUT_DIR", out):
keys = load_recent_shown_keys("2026-07-14", lookback_days=7)
self.assertEqual(keys["github_trending"], {"x/y"})
self.assertNotIn("a/b", keys["github_trending"])
def test_load_recent_falls_back_to_wecom_md_when_shown_missing(self):
"""旧日 data 无 wecom_shown_keys 时,从同日 wecom.md 解析实际展示 keys。"""
with tempfile.TemporaryDirectory() as tmp:
out = Path(tmp)
payload = {
"data": {
"date": "2026-07-13",
"github_trending": [{"repo": "other/top"}],
}
}
(out / "2026-07-13.data.json").write_text(
json.dumps(payload, ensure_ascii=False), encoding="utf-8"
)
(out / "2026-07-13.wecom.md").write_text(
"\n".join(
[
"🐙 **GitHub Trending Top 2**",
"1. [vinta/awesome-python](https://github.com/vinta/awesome-python)",
"2. [react/react](https://github.com/react/react)",
"",
"🌱 **GitHub 新兴 Top 1**",
"1. [elder-plinius/T3MP3ST](https://github.com/elder-plinius/T3MP3ST)",
]
),
encoding="utf-8",
)
with patch("daily.board_history.OUTPUT_DIR", out):
keys = load_recent_shown_keys("2026-07-14", lookback_days=7)
self.assertEqual(
keys["github_trending"],
{"vinta/awesome-python", "react/react"},
)
self.assertEqual(keys["github_emerging"], {"elder-plinius/T3MP3ST"})
self.assertNotIn("other/top", keys["github_trending"])
def test_merge_shown_does_not_touch_baseline(self):
data = {
"movement_baseline": {"github_trending": [{"repo": "raw/one"}]},
}
merged = merge_wecom_shown_into_data(
data, {"github_trending": ["shown/one"]}
)
self.assertEqual(
merged["movement_baseline"]["github_trending"][0]["repo"], "raw/one"
)
self.assertEqual(merged["wecom_shown_keys"]["github_trending"], ["shown/one"])
def test_persist_shown_keys_differs_from_baseline_keys(self):
from daily.board_select import board_select
raw = [{"repo": f"o/r{i}"} for i in range(10)]
recent = {f"o/r{i}" for i in range(3)}
selected = board_select(
board="github_trending",
items=raw,
recent_keys=recent,
limit=5,
pool_size=50,
kind="github",
)
baseline_keys = [x["repo"] for x in raw[:5]]
shown = extract_shown_keys("github_trending", selected)
data = {
"movement_baseline": {
"github_trending": [{"repo": k} for k in baseline_keys],
},
}
merged = merge_wecom_shown_into_data(data, {"github_trending": shown})
self.assertNotEqual(
set(merged["wecom_shown_keys"]["github_trending"]),
{x["repo"] for x in merged["movement_baseline"]["github_trending"]},
)
self.assertEqual(shown, ["o/r3", "o/r4", "o/r5", "o/r6", "o/r7"])
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,91 @@
# tests/test_board_select.py
from __future__ import annotations
import unittest
from daily.board_select import board_select
def _gh(repo: str) -> dict:
return {"repo": repo, "description": repo}
class BoardSelectTests(unittest.TestCase):
def test_filters_recent_and_keeps_order(self):
pool = [_gh(f"o/r{i}") for i in range(20)]
recent = {"o/r0", "o/r1", "o/r2"}
out = board_select(
board="github_trending",
items=pool,
recent_keys=recent,
limit=5,
pool_size=20,
kind="github",
)
keys = [x["repo"] for x in out]
self.assertEqual(keys, ["o/r3", "o/r4", "o/r5", "o/r6", "o/r7"])
def test_deep_pool_fills_after_filter(self):
pool = [_gh(f"o/r{i}") for i in range(8)]
recent = {f"o/r{i}" for i in range(6)} # 前 6 全封
out = board_select(
board="github_emerging",
items=pool,
recent_keys=recent,
limit=5,
pool_size=8,
kind="github",
)
self.assertEqual([x["repo"] for x in out], ["o/r6", "o/r7"]) # 短榜
def test_skill_uses_skill_id(self):
items = [
{"id": "a/b/s1", "source": "a/b", "title": "s1", "installs": 10},
{"id": "c/d/s2", "source": "c/d", "title": "s2", "installs": 9},
]
out = board_select(
board="skills_trending",
items=items,
recent_keys={"a/b/s1"},
limit=10,
pool_size=50,
kind="skill",
)
self.assertEqual([x["id"] for x in out], ["c/d/s2"])
def test_skill_filters_recent_by_source(self):
items = [
{"id": "a/b/s-new", "source": "a/b", "title": "s-new", "installs": 10},
{"id": "c/d/s2", "source": "c/d", "title": "s2", "installs": 9},
]
out = board_select(
board="skills_hot",
items=items,
recent_keys={"a/b"}, # source-level history
limit=10,
pool_size=50,
kind="skill",
)
self.assertEqual([x["id"] for x in out], ["c/d/s2"])
def test_skill_filters_recent_skill_id_as_same_source(self):
from daily.format_wecom import expand_skill_recent_keys
items = [
{"id": "open.feishu.cn/lark-drive", "source": "open.feishu.cn", "title": "lark-drive", "installs": 10},
{"id": "fresh/src/s", "source": "fresh/src", "title": "s", "installs": 9},
]
recent = expand_skill_recent_keys({"open.feishu.cn/lark-doc"})
out = board_select(
board="skills_trending",
items=items,
recent_keys=recent,
limit=10,
pool_size=50,
kind="skill",
)
self.assertEqual([x["id"] for x in out], ["fresh/src/s"])
if __name__ == "__main__":
unittest.main()

View File

@@ -1,15 +0,0 @@
from __future__ import annotations
def test_cursor_timeout_seconds_default(monkeypatch):
import daily.cursor_client as cc
monkeypatch.delenv("CURSOR_TIMEOUT", raising=False)
assert cc.cursor_timeout_seconds() == 600.0
def test_cursor_timeout_seconds_from_env(monkeypatch):
import daily.cursor_client as cc
monkeypatch.setenv("CURSOR_TIMEOUT", "900")
assert cc.cursor_timeout_seconds() == 900.0

View File

@@ -1,67 +0,0 @@
from __future__ import annotations
import json
import pytest
def test_build_movement_context_detects_new_skill(isolated_output, monkeypatch):
import daily.config as config
import daily.delta as delta
monkeypatch.setattr(config, "OUTPUT_DIR", isolated_output)
monkeypatch.setattr(delta, "OUTPUT_DIR", isolated_output)
prev_path = isolated_output / "2026-07-02.data.json"
prev_path.write_text(
json.dumps(
{
"data": {
"date": "2026-07-02",
"movement_baseline": {
"skills_trending": [{"id": "a/old", "title": "old"}],
"skills_hot": [],
"github_trending": [],
"github_emerging": [],
"github_topic": [],
},
}
},
ensure_ascii=False,
),
encoding="utf-8",
)
trending = [
{"id": "a/new", "title": "new-skill", "source": "a", "installs": 10, "link": "https://x", "description": ""},
{"id": "a/old", "title": "old", "source": "a", "installs": 9, "link": "https://y", "description": ""},
]
movement = delta.build_movement_context(
date_str="2026-07-03",
trending=trending,
hot=[],
github_trending=[],
github_emerging=[],
github_topic=[],
)
assert movement["baseline_date"] == "2026-07-02"
assert len(movement["skills_trending_moves"]) == 1
assert movement["skills_trending_moves"][0]["id"] == "a/new"
def test_find_previous_data_skips_missing_days(isolated_output, monkeypatch):
import daily.config as config
import daily.delta as delta
monkeypatch.setattr(config, "OUTPUT_DIR", isolated_output)
monkeypatch.setattr(delta, "OUTPUT_DIR", isolated_output)
(isolated_output / "2026-07-01.data.json").write_text(
json.dumps({"data": {"date": "2026-07-01", "skills_trending": []}}),
encoding="utf-8",
)
found = delta.find_previous_data("2026-07-03")
assert found is not None
assert found[0] == "2026-07-01"

View File

@@ -0,0 +1,95 @@
"""Tests for daily.featured_pick."""
from __future__ import annotations
import os
import unittest
from unittest.mock import patch
from daily.featured_pick import (
apply_featured_pick,
match_in_data,
parse_featured_pick,
pick_command_from_featured,
pick_why_from_featured,
research_featured_pick,
)
SAMPLE_INPUT = {
"skills_trending": [
{
"id": "foo/bar/gstack-cli",
"title": "gstack-cli",
"source": "foo/bar",
"installs": 1200,
"installs_fmt": "1.2K",
"link": "https://skills.sh/foo/bar/gstack-cli",
"description": "Agent workflow CLI",
}
],
"skills_hot": [],
"github_trending": [
{
"repo": "acme/gstack",
"url": "https://github.com/acme/gstack",
"total_stars_fmt": "3.2K",
"description": "GStack toolkit",
}
],
"github_emerging": [],
"github_topic": {"topic": "llm", "repos": []},
}
class ParseFeaturedPickTests(unittest.TestCase):
def test_empty(self):
with patch.dict(os.environ, {}, clear=True):
self.assertIsNone(parse_featured_pick())
def test_query_only(self):
with patch.dict(os.environ, {"DAILY_FEATURED_PICK": "gstack"}, clear=True):
self.assertEqual(parse_featured_pick(), {"query": "gstack"})
def test_query_with_url(self):
with patch.dict(os.environ, {"DAILY_FEATURED_PICK": "gstack|https://example.com"}, clear=True):
self.assertEqual(
parse_featured_pick(),
{"query": "gstack", "url_hint": "https://example.com"},
)
class MatchInDataTests(unittest.TestCase):
def test_matches_skill_and_github(self):
matches = match_in_data(SAMPLE_INPUT, "gstack")
self.assertEqual(len(matches["skills"]), 1)
self.assertEqual(matches["skills"][0]["title"], "gstack-cli")
self.assertEqual(len(matches["github"]), 1)
self.assertEqual(matches["github"][0]["repo"], "acme/gstack")
class FeaturedPickWorkflowTests(unittest.TestCase):
def test_fallback_without_llm(self):
llm_input = dict(SAMPLE_INPUT)
with patch.dict(os.environ, {"DAILY_FEATURED_PICK": "gstack"}, clear=True):
with patch("daily.featured_pick.has_llm_configured", return_value=False):
featured = research_featured_pick(llm_input, date_str="2026-07-03")
self.assertIsNotNone(featured)
assert featured is not None
self.assertEqual(featured["type"], "skill")
self.assertIn("npx skills add foo/bar/gstack-cli", featured["command"])
self.assertTrue(featured["why_today"])
def test_apply_featured_pick_mutates_input(self):
llm_input = dict(SAMPLE_INPUT)
with patch.dict(os.environ, {"DAILY_FEATURED_PICK": "gstack"}, clear=True):
with patch("daily.featured_pick.has_llm_configured", return_value=False):
featured = apply_featured_pick(llm_input, date_str="2026-07-03")
self.assertIsNotNone(featured)
self.assertIn("featured_pick", llm_input)
self.assertEqual(pick_command_from_featured(featured), llm_input["featured_pick"]["command"])
self.assertEqual(pick_why_from_featured(featured), llm_input["featured_pick"]["why_today"])
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,55 @@
# tests/test_featured_reason.py
"""T6: 首推理由行(pick_why + DAILY_FEATURED_REASON 开关)测试。"""
from __future__ import annotations
import unittest
from unittest import mock
import daily.format_wecom as fw
def _build(pick_why: str) -> str:
return fw.build_wecom_report(
date_str="2026-07-18",
time_str="08:50 (UTC+8)",
updated="2026-07-18",
highlights=[],
theme_line="**今日主题**:测试",
ai_news=None,
cn_ai_news=None,
merged_ai_news=None,
merged_tech_ai_news=None,
trending=[],
hot=[],
repos=[],
emerging=[],
topic_name="t",
topic_repos=[],
pick_command="npx skills add src/alpha",
pick_why=pick_why,
pick_title="alpha",
pick_url="https://x/a",
include_boards=False,
)
class FeaturedReasonTests(unittest.TestCase):
def test_reason_shown_when_present_and_enabled(self):
with mock.patch.object(fw, "env_bool", return_value=True):
md = _build("昨日 star 增速第一")
self.assertIn("> 昨日 star 增速第一", md)
def test_reason_hidden_when_switch_off(self):
with mock.patch.object(fw, "env_bool", return_value=False):
md = _build("昨日 star 增速第一")
self.assertNotIn("> 昨日 star 增速第一", md)
def test_reason_omitted_when_empty(self):
with mock.patch.object(fw, "env_bool", return_value=True):
md = _build("")
self.assertIn("今日首推", md)
self.assertNotIn("> \n", md)
if __name__ == "__main__":
unittest.main()

Some files were not shown because too many files have changed in this diff Show More