ftb-diy/daily-robots
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: 早报系统重构与功能增强

Browse Source 
- 新增常驻调度器 daily/scheduler.py + run-scheduler.ps1（定时生成/推送）
- 新增 daily/bridge_manager.py：Windows 兼容的 Cursor SDK 桥接
- 新增 skills/daily-featured-pick 首推 Skill 与叙事轴/去重逻辑
- 新闻抓取窗口、GitHub 搜索、企微 delta 模式等多项改进
- 补充设计文档与 superpowers 计划/规范
- 新增对应测试（scheduler、featured_pick、github_search、news_fetch_window 等）

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
王礼
2026-07-17 18:12:00 +08:00
parent 6192dd4e2a
commit 6ea2a4e4c6
35 changed files with 4389 additions and 359 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
.env.example
View File

@@ -25,12 +25,12 @@ GITHUB_TRENDING_SINCE=daily
# GITHUB_TOKEN=ghp_xxxxxxxxxxxxxxxx # GITHUB_TOKEN=ghp_xxxxxxxxxxxxxxxx
# GITHUB_API_ENRICH=1 # GITHUB_API_ENRICH=1
# 企微短版（各榜 Top N，默认 10） # 企微短版（各榜 Top N，默认 5；周内不重复见 DAILY_BOARD_DEDUP_DAYS）
DAILY_WECOM_TRENDING=10 DAILY_WECOM_TRENDING=5
DAILY_WECOM_HOT=10 DAILY_WECOM_HOT=5
DAILY_WECOM_GITHUB_TRENDING=10 DAILY_WECOM_GITHUB_TRENDING=5
DAILY_WECOM_GITHUB_EMERGING=10 DAILY_WECOM_GITHUB_EMERGING=5
DAILY_WECOM_GITHUB_TOPIC=10 DAILY_WECOM_GITHUB_TOPIC=5
DAILY_WECOM_AI_NEWS=10 DAILY_WECOM_AI_NEWS=10
# research 模式额外技术类时讯条数（叠加在 AI 时讯精选之上） # research 模式额外技术类时讯条数（叠加在 AI 时讯精选之上）
DAILY_WECOM_AI_NEWS_TECH=5 DAILY_WECOM_AI_NEWS_TECH=5
@@ -105,7 +105,7 @@ DAILY_AI_NEWS_PER_FEED=3
DAILY_AI_NEWS_PER_CATEGORY=5 DAILY_AI_NEWS_PER_CATEGORY=5
# 多样性 / 去重（见 docs/superpowers/specs/2026-07-14-wecom-diversity-dedup-design.md） # 多样性 / 去重（见 docs/superpowers/specs/2026-07-14-wecom-diversity-dedup-design.md）
# DAILY_BOARD_DEDUP_DAYS=7 DAILY_BOARD_DEDUP_DAYS=7
# DAILY_BOARD_POOL_SIZE=50 # DAILY_BOARD_POOL_SIZE=50
# DAILY_FEATURED_DEDUP_DAYS=30 # DAILY_FEATURED_DEDUP_DAYS=30
# DAILY_THEME_BAN_DAYS=7 # DAILY_THEME_BAN_DAYS=7

202
README.md
View File

@@ -1,6 +1,6 @@
# skills-hot-daily # skills-hot-daily
Skills / GitHub 早报推送 + 企微对话机器人（同一仓库、两套企微接入）。 Skills / GitHub 早报推送（企微 Webhook）。
## 项目结构 ## 项目结构
@@ -8,7 +8,7 @@ Skills / GitHub 早报推送 + 企微对话机器人（同一仓库、两套企
skills-hot-daily/ skills-hot-daily/
├── README.md ├── README.md
├── .env.example # 早报 webhook、GitHub 等 ├── .env.example # 早报 webhook、GitHub 等
├── requirements.txt # 早报 Python 依赖 ├── requirements.txt # Python 依赖
├── run-daily.ps1 # 生成 + 推送一条龙 ├── run-daily.ps1 # 生成 + 推送一条龙
├── send-wecom.ps1 # 仅推送 ├── send-wecom.ps1 # 仅推送
├── daily/ # 早报 Python 包 ├── daily/ # 早报 Python 包
@@ -32,24 +32,15 @@ skills-hot-daily/
├── skills/daily-editor/ # 早报 Cursor 编辑 Skill ├── skills/daily-editor/ # 早报 Cursor 编辑 Skill
│ └── SKILL.md │ └── SKILL.md
├── logs/ ├── logs/
├── .cache/ └── .cache/
└── bot/ # 企微 API 模式对话机器人（独立 venv）
├── main.py
├── skills_service.py
└── scenarios/
``` ```
| 模块 | 配置文件 | 启动方式 |
|------|----------|----------|
| **早报推送** | 根目录 `.env`（`WECOM_WEBHOOK_KEY` 等） | `.\run-daily.ps1` |
| **对话 Bot** | `bot/.env`（`WECOM_BOT_ID` / `SECRET` 等） | `cd bot` → `python main.py` |
--- ---
## 一、早报推送 ## 早报推送
```powershell ```powershell
cd d:\LY\test\tech\skills-hot-daily cd d:\LY\diy\daily-robots
pip install -r requirements.txt pip install -r requirements.txt
copy .env.example .env copy .env.example .env
.\run-daily.ps1 .\run-daily.ps1
@@ -84,15 +75,14 @@ WECOM_WEBHOOK_KEY=your-key
| 研究 / 论文 | arXiv cs.CL/AI/LG、HF Papers | | 研究 / 论文 | arXiv cs.CL/AI/LG、HF Papers |
| 社区讨论 | HN、Reddit r/LocalLLaMA / ClaudeAI / ML 等 | | 社区讨论 | HN、Reddit r/LocalLLaMA / ClaudeAI / ML 等 |
环境变量：`DAILY_AI_NEWS=1` · `DAILY_CN_AI_NEWS=1` · `DAILY_AI_NEWS_HOURS=72` · `DAILY_WECOM_AI_NEWS=10` · `DAILY_WECOM_CN_AI_NEWS=8` 环境变量：`DAILY_AI_NEWS=1` · `DAILY_CN_AI_NEWS=1` · `DAILY_AI_NEWS_HOURS=24` · `DAILY_WECOM_AI_NEWS=10` · `DAILY_WECOM_CN_AI_NEWS=10`
**国内 AI 时讯**（RSS，见 `daily/news/feeds_cn.py`）： **国内 AI 时讯**（RSS，见 `daily/news/feeds_cn.py`）：
| 类别 | 覆盖 | | 类别 | 覆盖 |
|------|------| |------|------|
| AI 专业媒体 | 量子位、InfoQ 中文 | | AI 专业媒体 | 量子位 |
| 综合科技 | 36氪、雷锋网、Google News 中文 | | 综合科技 | 36氪、雷锋网、Google News 中文 |
| 开发者社区 | 掘金（标题 AI 关键词过滤） |
### 生成架构（Tier B · Cursor 编辑层） ### 生成架构（Tier B · Cursor 编辑层）
@@ -135,7 +125,9 @@ Python 抓取 → Step1 趋势分析 → Step2 叙事写稿 → Python 分条推
```env ```env
DAILY_REPORT_MODE=agent DAILY_REPORT_MODE=agent
DAILY_CURSOR_CWD=d:\LY\diy\skills-hot-daily # 早报 LLM 工作目录（与 bot 的 CURSOR_CWD 独立） CURSOR_API_KEY=cursor_...
CURSOR_MODEL=composer-2.5
DAILY_CURSOR_CWD=d:\LY\diy\daily-robots
``` ```
| 文件 | 说明 | | 文件 | 说明 |
@@ -146,169 +138,11 @@ DAILY_CURSOR_CWD=d:\LY\diy\skills-hot-daily # 早报 LLM 工作目录（与 bo
- 完整版 `YYYY-MM-DD.md` 仍为数据表格归档；企微版由 Agent 直接写 Markdown - 完整版 `YYYY-MM-DD.md` 仍为数据表格归档；企微版由 Agent 直接写 Markdown
- Agent 失败自动回退 `classic`，不影响 `run-daily.ps1` - Agent 失败自动回退 `classic`，不影响 `run-daily.ps1`
定时推送：Windows 任务计划程序或 `/loop 1d` 执行 `run-daily.ps1`。 定时推送：
--- - **常驻调度（推荐）**：`python -m daily schedule` 或 `.\run-scheduler.ps1`（默认 08:50 生成、09:00 推送，见 `DAILY_SCHEDULE_*`）
- Windows 任务计划：`.\register-daily-task.ps1`
## 二、可对话 Skills 助手（企业微信智能机器人） - Cursor：`/loop 1d`（时间会漂移，仅临时用）
在企微里 @ 机器人即可：
- **快查**：`trending 10`、`hot 10`、`搜索 react`（本地 skills 数据，秒回）
- **截图预览**：`preview` / `截图`（基于 `.env` 的 `CURSOR_CWD` 启动前端并发图）
- **通用任务**：任意自然语言需求，由 **Cursor Agent** 执行并回传结果
### 1. 创建 API 模式机器人
1. [企业微信管理后台](https://work.weixin.qq.com/) → **安全与管理** → **管理工具** → **智能机器人** → **创建机器人**
2. 选择 **API 模式创建** → **使用长连接**
3. 记录 **Bot ID** 和 **Secret**（Secret 只显示一次，请立即保存）
4. 设置可见范围，将机器人 **添加到目标群** 或允许成员单聊
普通成员路径：工作台 → 智能机器人 → 手动创建 → API 模式 → 长连接
### 2. 启动本地服务
```powershell
cd d:\LY\test\tech\skills-hot-daily\bot
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
playwright install chromium
copy .env.example .env
# 编辑 .env：WECOM_BOT_ID / WECOM_BOT_SECRET / CURSOR_API_KEY
python main.py
```
服务需 **常驻运行**（本机、服务器或 Docker）。长连接模式下机器人进程须在线才能收消息。
### 3. 路由模式（ROUTING_MODE）
| 模式 | 行为 |
|------|------|
| `hybrid`（默认） | `trending`/`hot`/`搜索`/`详情` 走本地快查；其余 @ 消息交给 Cursor |
| `cursor` | 所有消息都交给 Cursor 执行 |
| `skills` | 仅本地 skills 快查（旧行为） |
**Cursor 任务示例**（群里发送）：
```
@test 总结 trending top10，并推荐 3 个适合前端团队的 skill
@test 对比 mattpocock/skills 和 obra/superpowers 各有哪些热门 skill
@test 帮我写一段 npx skills add 的安装说明
```
Cursor 在本机 `CURSOR_CWD` 目录下运行，默认 `d:\LY\test\tech`。复杂任务可能需要 1–10 分钟，流式消息会显示「Cursor 正在执行任务…」。
### 4. 前端截图预览（API 模式发图）
项目路径读取 `.env` 中的 **`CURSOR_CWD`**。机器人会：
1. 在 `CURSOR_CWD` 检测 `package.json`，若有 `dev` 脚本则执行 `PREVIEW_DEV_COMMAND`（默认 `npm run dev`）
2. 等待 `PREVIEW_PORT`（默认 `5173`）就绪，或用 `PREVIEW_URL` 直接访问
3. Playwright 打开页面并截图
4. 通过 API 模式 **上传图片 + 回复 image 消息** 到群
| 命令 | 说明 |
|------|------|
| `preview` / `截图` / `预览` | 访问 `http://127.0.0.1:5173/` 并截图 |
| `preview /login` | 指定路径 |
| `preview / 3000` | 指定端口 |
| `preview http://127.0.0.1:8080/` | 指定完整 URL |
**多步网页操作**（登录、点菜单、再截图）见下一节，不再写死在代码里。
`.env` 可选配置：
```env
CURSOR_CWD=d:\LY\test\tech
PREVIEW_PORT=5173
PREVIEW_URL=http://127.0.0.1:5173/
PREVIEW_DEV_COMMAND=npm run dev
PREVIEW_STARTUP_TIMEOUT=120
```
若 `CURSOR_CWD` 下暂无前端项目，可先手动启动 dev server，或设置 `PREVIEW_URL` 指向已运行地址。
### 4b. 网页操作（Playwright 步骤引擎）
支持三种方式定义操作流程，**无需改 Python 代码**：
**1. 自然语言（企微里直接说）**
```
@test 访问登录页，输入账号密码，点击登录后进入主页，点击智能体管理菜单然后截图
```
账号密码从 `.env` 读取（`{{PREVIEW_LOGIN_USER}}` / `{{PREVIEW_LOGIN_PASSWORD}}`），勿在群里发密码。
**2. 场景文件 YAML**
`bot/scenarios/xiaobao-agent-manage.yaml` 示例：
```yaml
name: xiaobao-agent-manage
steps:
- goto: /login
- fill:
field: 账号
value: "{{PREVIEW_LOGIN_USER}}"
- fill:
field: 密码
value: "{{PREVIEW_LOGIN_PASSWORD}}"
- click: 登录
- wait:
url: "**/app/**"
- click: 智能体管理
- wait: 1500
- screenshot
```
触发：`@test browser xiaobao-agent-manage`
场景搜索路径：`bot/scenarios/`、`CURSOR_CWD/.browser-scenarios/`、环境变量 `BROWSER_SCENARIOS_DIR`。
**3. 消息内 DSL**
```
browser:
goto /login
fill 账号 {{PREVIEW_LOGIN_USER}}
fill 密码 {{PREVIEW_LOGIN_PASSWORD}}
click 登录
click 智能体管理
screenshot
```
**支持的步骤**：`goto` · `fill` · `click` · `wait` · `screenshot` · `press`
`.env` 登录与场景配置：
```env
PREVIEW_LOGIN_USER=test_account
PREVIEW_LOGIN_PASSWORD=your_password
# BROWSER_DEFAULT_SCENARIO=xiaobao-agent-manage
```
### 5. 支持的快查命令
| 命令 | 说明 |
|------|------|
| `trending 10` / `趋势 10` | 近期增长榜 Top N（默认 10，最大 30） |
| `hot 10` / `实时 10` | 实时热度榜 |
| `all 10` / `总榜 10` | 历史总安装榜 |
| `搜索 react` / `search tdd` | 关键词搜索 |
| `详情 find-skills` | 单个 skill 详情 + 安装命令 |
| `preview` / `截图` | 启动 CURSOR_CWD 前端并截图发群 |
| `帮助` | 命令列表 |
自然语言（非显式快查命令）会交给 **Cursor** 处理，例如 `@test 查 trending 并写推荐` 。
### 6. 本地测试（无需企微凭证）
```powershell
cd d:\LY\test\tech\skills-hot-daily\bot
python -c "from skills_service import handle_command; print(handle_command('trending 5'))"
```
--- ---
@@ -321,14 +155,6 @@ python -c "from skills_service import handle_command; print(handle_command('tren
| **邮件 + 企业微信邮箱** | 已有 SMTP | 中 | | **邮件 + 企业微信邮箱** | 已有 SMTP | 中 |
| **PushPlus / Server酱** | 个人微信中转 | 低（第三方） | | **PushPlus / Server酱** | 个人微信中转 | 低（第三方） |
### 应用消息 API（简要）
适合「推送给某个人」而非群聊。需在 [企业微信管理后台](https://work.weixin.qq.com/) 创建自建应用，调用：
`POST https://qyapi.weixin.qq.com/cgi-bin/message/send?access_token=TOKEN`
消息体支持 `text` / `markdown` / `news` 等。需先 `gettoken` 再发消息，并维护 access_token 缓存。
--- ---
## 注意事项 ## 注意事项

10
daily/__main__.py
View File

@@ -5,6 +5,7 @@ from __future__ import annotations
import sys import sys
from daily.generate import main as generate_main from daily.generate import main as generate_main
from daily.scheduler import main as schedule_main
from daily.webhook import main as push_main from daily.webhook import main as push_main
@@ -14,7 +15,14 @@ def main() -> int:
return generate_main() return generate_main()
if cmd in {"push", "send", "webhook"}: if cmd in {"push", "send", "webhook"}:
return push_main(sys.argv[2:]) return push_main(sys.argv[2:])
print(f"未知命令: {cmd}\n用法: python -m daily [generate|push] [report_path]", file=sys.stderr) if cmd in {"schedule", "scheduler", "daemon"}:
return schedule_main()
print(
f"未知命令: {cmd}\n"
"用法: python -m daily [generate|push|schedule] [report_path]\n"
" python -m daily schedule [--once] [--dry-run]",
file=sys.stderr,
)
return 1 return 1

124
daily/board_history.py
View File

@@ -4,6 +4,7 @@ from __future__ import annotations
import json import json
import logging import logging
import re
from datetime import datetime, timedelta from datetime import datetime, timedelta
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
@@ -15,29 +16,119 @@ logger = logging.getLogger(__name__)
BOARD_KEYS = RECENT_BOARD_KEYS BOARD_KEYS = RECENT_BOARD_KEYS
_GITHUB_REPO_RE = re.compile(r"github\.com/([\w.-]+/[\w.-]+)", re.I)
_SKILL_SH_RE = re.compile(r"skills\.sh/([\w.-]+/[\w.-]+(?:/[\w.-]+)?)", re.I)
# 与企微正文榜单标题对齐；顺序用于切分相邻 section
_WECOM_SECTION_PATTERNS: tuple[tuple[str, re.Pattern[str]], ...] = (
("skills_trending", re.compile(r"Skills\s+Trending", re.I)),
("skills_hot", re.compile(r"Skills\s+Hot", re.I)),
("github_trending", re.compile(r"GitHub\s+Trending", re.I)),
("github_emerging", re.compile(r"GitHub\s+新兴", re.I)),
("github_topic", re.compile(r"Topic\s+", re.I)),
)
def extract_shown_keys(board: str, items: list[dict[str, Any]]) -> list[str]: def extract_shown_keys(board: str, items: list[dict[str, Any]]) -> list[str]:
"""从最终展示 items 抽取稳定 identity key。""" """从最终展示 items 抽取稳定 identity key。
Skills 榜同时写入 skill id 与 source，便于周去重按仓屏蔽。
"""
keys: list[str] = [] keys: list[str] = []
seen: set[str] = set() seen: set[str] = set()
for item in items: for item in items:
if board.startswith("skills_"): if board.startswith("skills_"):
key = skill_id(item) candidates = [skill_id(item), str(item.get("source") or "").strip()]
else: else:
key = str(item.get("repo") or "") candidates = [str(item.get("repo") or "")]
if not key or key in seen: for key in candidates:
continue if not key or key in seen:
seen.add(key) continue
keys.append(key) seen.add(key)
keys.append(key)
return keys return keys
def _keys_from_board_items(board: str, data: dict[str, Any]) -> set[str]:
if board == "github_topic":
topic = data.get("github_topic") or {}
items = topic.get("repos") if isinstance(topic, dict) else []
else:
items = data.get(board) or []
if not isinstance(items, list):
return set()
return set(extract_shown_keys(board, items))
def parse_wecom_shown_keys(md: str) -> dict[str, set[str]]:
"""从企微 Markdown 按榜单 section 解析已展示 keys（冷启动兼容）。"""
out: dict[str, set[str]] = {board: set() for board in BOARD_KEYS}
if not (md or "").strip():
return out
hits: list[tuple[int, str]] = []
for board, pattern in _WECOM_SECTION_PATTERNS:
for match in pattern.finditer(md):
hits.append((match.start(), board))
if not hits:
return out
hits.sort(key=lambda x: x[0])
for idx, (start, board) in enumerate(hits):
end = hits[idx + 1][0] if idx + 1 < len(hits) else len(md)
chunk = md[start:end]
if board.startswith("skills_"):
out[board].update(_SKILL_SH_RE.findall(chunk))
else:
out[board].update(_GITHUB_REPO_RE.findall(chunk))
return out
def _load_shown_keys_for_day(path: Path, date_str: str) -> dict[str, set[str]] | None:
"""读一日历史：优先 wecom_shown_keys；缺省则回退 wecom.md，再回退 data 榜字段。"""
empty = {board: set() for board in BOARD_KEYS}
try:
payload = json.loads(path.read_text(encoding="utf-8"))
except (OSError, ValueError) as exc:
logger.warning("读取 wecom_shown_keys %s 失败：%s", path, exc)
return None
data = payload.get("data") if isinstance(payload, dict) else None
if not isinstance(data, dict):
return empty
out: dict[str, set[str]] = {board: set() for board in BOARD_KEYS}
shown = data.get("wecom_shown_keys")
if isinstance(shown, dict):
for board in BOARD_KEYS:
keys = shown.get(board) or []
if isinstance(keys, list):
out[board].update(str(k) for k in keys if k)
if any(out.values()):
return out
wecom_path = OUTPUT_DIR / f"{date_str}.wecom.md"
if wecom_path.exists():
try:
md = wecom_path.read_text(encoding="utf-8")
except OSError as exc:
logger.warning("读取 wecom.md 回退 %s 失败：%s", wecom_path, exc)
else:
parsed = parse_wecom_shown_keys(md)
if any(parsed.values()):
return parsed
for board in BOARD_KEYS:
out[board].update(_keys_from_board_items(board, data))
return out
def load_recent_shown_keys( def load_recent_shown_keys(
date_str: str, date_str: str,
*, *,
lookback_days: int | None = None, lookback_days: int | None = None,
) -> dict[str, set[str]]: ) -> dict[str, set[str]]:
"""近 N 日 data.wecom_shown_keys 并集（不含当日）。缺省或读失败视为空集。""" """近 N 日已展示 keys 并集（不含当日）。缺省或读失败视为空集。"""
empty = {board: set() for board in BOARD_KEYS} empty = {board: set() for board in BOARD_KEYS}
try: try:
dt = datetime.strptime(date_str, "%Y-%m-%d") dt = datetime.strptime(date_str, "%Y-%m-%d")
@@ -52,22 +143,11 @@ def load_recent_shown_keys(
path = OUTPUT_DIR / f"{prev_date}.data.json" path = OUTPUT_DIR / f"{prev_date}.data.json"
if not path.exists(): if not path.exists():
continue continue
try: day_keys = _load_shown_keys_for_day(path, prev_date)
payload = json.loads(path.read_text(encoding="utf-8")) if day_keys is None:
except (OSError, ValueError) as exc:
logger.warning("读取 wecom_shown_keys %s 失败：%s", path, exc)
continue
data = payload.get("data")
if not isinstance(data, dict):
continue
shown = data.get("wecom_shown_keys")
if not isinstance(shown, dict):
continue continue
for board in BOARD_KEYS: for board in BOARD_KEYS:
keys = shown.get(board) or [] out[board].update(day_keys.get(board) or set())
if not isinstance(keys, list):
continue
out[board].update(str(k) for k in keys if k)
return out return out

20
daily/board_select.py
View File

@@ -24,20 +24,22 @@ def board_select(
from daily.skills_group import group_skills_by_source from daily.skills_group import group_skills_by_source
pool = group_skills_by_source(items, limit=pool_size, pool_size=pool_size) pool = group_skills_by_source(items, limit=pool_size, pool_size=pool_size)
def key_fn(x: dict[str, Any]) -> str:
return skill_id(x)
else: else:
pool = items[: max(pool_size, limit)] pool = items[: max(pool_size, limit)]
def key_fn(x: dict[str, Any]) -> str:
return str(x.get("repo") or "")
out: list[dict[str, Any]] = [] out: list[dict[str, Any]] = []
for item in pool: for item in pool:
k = key_fn(item) if kind == "skill":
if not k or k in recent_keys: key = skill_id(item)
continue source = str(item.get("source") or "").strip()
if (key and key in recent_keys) or (source and source in recent_keys):
continue
if not key and not source:
continue
else:
key = str(item.get("repo") or "")
if not key or key in recent_keys:
continue
out.append(item) out.append(item)
if len(out) >= limit: if len(out) >= limit:
break break

156
daily/bridge_manager.py Normal file
View File

@@ -0,0 +1,156 @@
"""Windows 兼容的 Cursor SDK bridge 管理。"""
from __future__ import annotations
import codecs
import json
import logging
import os
import subprocess
import threading
import time
from pathlib import Path
from typing import Any, Mapping
from daily.config import ROOT, env
logger = logging.getLogger(__name__)
READY_LINE_PREFIX = "cursor-sdk-bridge ready "
_bridge_lock = threading.Lock()
_bridge_process: subprocess.Popen[bytes] | None = None
def _cursor_cwd() -> str:
return env("DAILY_CURSOR_CWD") or env("CURSOR_CWD") or str(ROOT)
def _parse_discovery_line(line: str) -> Mapping[str, Any] | None:
if not line.startswith(READY_LINE_PREFIX):
return None
payload = line[len(READY_LINE_PREFIX) :].strip()
loaded = json.loads(payload)
if not isinstance(loaded, dict):
raise RuntimeError("Bridge discovery payload must be an object")
return loaded
def _read_discovery_polling(process: subprocess.Popen[bytes], timeout: float = 60) -> Mapping[str, Any]:
"""不用 selectors，避免 Windows 上 WinError 10038。"""
if process.stderr is None:
raise RuntimeError("Bridge stderr unavailable")
fd = process.stderr.fileno()
was_blocking = os.get_blocking(fd)
os.set_blocking(fd, False)
decoder = codecs.getincrementaldecoder("utf-8")(errors="replace")
pending = ""
stderr_lines: list[str] = []
deadline = time.monotonic() + timeout
try:
while time.monotonic() < deadline:
try:
chunk = os.read(fd, 8192)
except BlockingIOError:
chunk = b""
if chunk:
pending += decoder.decode(chunk)
while "\n" in pending:
line, pending = pending.split("\n", 1)
stderr_lines.append(line)
discovery = _parse_discovery_line(line)
if discovery is not None:
return discovery
else:
code = process.poll()
if code is not None:
pending += decoder.decode(b"", final=True)
if pending.strip():
stderr_lines.append(pending.strip())
joined = "\n".join(stderr_lines)[-2000:]
raise RuntimeError(
f"Bridge 启动失败 exit={code}: {joined or '无 stderr 输出'}"
)
time.sleep(0.05)
finally:
os.set_blocking(fd, was_blocking)
raise RuntimeError("等待 Cursor bridge 就绪超时")
def _auth_token_from_discovery(discovery: Mapping[str, Any]) -> str:
token = str(discovery.get("authToken") or "").strip()
if token:
return token
token_file = discovery.get("authTokenFile")
if token_file:
return Path(str(token_file)).read_text(encoding="utf-8").strip()
raise RuntimeError("Bridge discovery 缺少 auth token")
def warm_cursor_bridge(force: bool = False) -> None:
"""启动 cursor-sdk-bridge 并写入 CURSOR_SDK_BRIDGE_* 环境变量。"""
global _bridge_process
with _bridge_lock:
if (
not force
and _bridge_process is not None
and _bridge_process.poll() is None
and os.environ.get("CURSOR_SDK_BRIDGE_URL")
and os.environ.get("CURSOR_SDK_BRIDGE_TOKEN")
):
return
if _bridge_process is not None and _bridge_process.poll() is None:
_bridge_process.terminate()
try:
_bridge_process.wait(timeout=5)
except subprocess.TimeoutExpired:
_bridge_process.kill()
from cursor_sdk._vendor import resolve_bridge_path
cwd = _cursor_cwd()
argv = [resolve_bridge_path(), "--workspace", cwd]
logger.info("启动 Cursor bridge workspace=%s", cwd)
process = subprocess.Popen(
argv,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
)
try:
discovery = _read_discovery_polling(process)
except Exception:
process.kill()
process.wait(timeout=5)
raise
url = str(discovery.get("url") or "").strip()
if not url:
host = str(discovery.get("host") or "127.0.0.1")
port = discovery.get("port")
url = f"http://{host}:{port}"
token = _auth_token_from_discovery(discovery)
os.environ["CURSOR_SDK_BRIDGE_URL"] = url
os.environ["CURSOR_SDK_BRIDGE_TOKEN"] = token
_bridge_process = process
logger.info("Cursor bridge 就绪: %s", url)
def shutdown_cursor_bridge() -> None:
global _bridge_process
with _bridge_lock:
if _bridge_process is None:
return
if _bridge_process.poll() is None:
_bridge_process.terminate()
try:
_bridge_process.wait(timeout=5)
except subprocess.TimeoutExpired:
_bridge_process.kill()
_bridge_process = None

4
daily/config.py
View File

@@ -149,8 +149,8 @@ def board_dedup_days() -> int:
def board_pool_size() -> int: def board_pool_size() -> int:
fallback = env_int("DAILY_WECOM_SKILL_POOL", 50) fallback = env_int("DAILY_WECOM_SKILL_POOL", 400)
return max(1, env_int("DAILY_BOARD_POOL_SIZE", max(50, fallback))) return max(1, env_int("DAILY_BOARD_POOL_SIZE", max(200, fallback)))
def featured_dedup_days() -> int: def featured_dedup_days() -> int:

11
daily/featured_pick.py
View File

@@ -62,6 +62,10 @@ def load_recent_featured_keys(date_str: str, days: int | None = None) -> set[str
if not isinstance(data, dict): if not isinstance(data, dict):
continue continue
key = str(data.get("featured_pick_key") or "").strip() key = str(data.get("featured_pick_key") or "").strip()
if not key:
featured = data.get("featured_pick")
if isinstance(featured, dict):
key = featured_identity_key(featured)
if key: if key:
out.add(key) out.add(key)
return out return out
@@ -84,7 +88,12 @@ def load_yesterday_featured_key(date_str: str) -> str | None:
if not isinstance(data, dict): if not isinstance(data, dict):
return None return None
key = str(data.get("featured_pick_key") or "").strip() key = str(data.get("featured_pick_key") or "").strip()
return key or None if key:
return key
featured = data.get("featured_pick")
if isinstance(featured, dict):
return featured_identity_key(featured) or None
return None
def _featured_rng(date_str: str) -> random.Random: def _featured_rng(date_str: str) -> random.Random:

224
daily/format_wecom.py
View File

@@ -7,7 +7,7 @@ from typing import Any
from daily.config import wecom_skill_desc_limit from daily.config import wecom_skill_desc_limit
from daily.localize import LocalizeJob, localize_brief_descriptions, needs_chinese from daily.localize import LocalizeJob, localize_brief_descriptions, needs_chinese
from daily.skills_group import group_skills_by_source, skill_id as board_skill_id from daily.skills_group import group_skills_by_source
from daily.text_utils import trim_brief from daily.text_utils import trim_brief
ICONS = { ICONS = {
@@ -469,6 +469,32 @@ def _prepare_grouped_wecom_skills(
return wecom_items[:limit], keys return wecom_items[:limit], keys
def _normalize_skill_source_groups(items: list[dict[str, Any]]) -> list[dict[str, Any]]:
"""将条目规范为按 source 合并的榜单项（展示始终为合并态）。"""
flat: list[dict[str, Any]] = []
for item in items:
flat.extend(_flatten_skill_board_item(item))
if not flat:
return []
return group_skills_by_source(flat, limit=len(flat), pool_size=len(flat))
def _skill_primary_id(item: dict[str, Any]) -> str:
return str(item.get("id") or f"{item.get('source')}/{item.get('title')}" or "").strip()
def _source_from_skill_key(key: str) -> str:
from daily.skills_group import source_from_skill_key
return source_from_skill_key(key)
def expand_skill_recent_keys(keys: set[str] | None) -> set[str]:
from daily.skills_group import expand_skill_recent_keys as _expand
return _expand(keys)
def _merge_skill_board_items( def _merge_skill_board_items(
moves: list[dict[str, Any]], moves: list[dict[str, Any]],
full_items: list[dict[str, Any]], full_items: list[dict[str, Any]],
@@ -477,29 +503,80 @@ def _merge_skill_board_items(
exclude_keys: set[str] | None = None, exclude_keys: set[str] | None = None,
recent_keys: set[str] | None = None, recent_keys: set[str] | None = None,
) -> tuple[list[dict[str, Any]], set[str]]: ) -> tuple[list[dict[str, Any]], set[str]]:
"""异动优先，不足时用深池补满；按 source 合并态取条。
周去重按 source（含从 skill id 展开）；同日避开其它榜时也按 source。
"""
from daily.delta import skill_id as move_skill_id from daily.delta import skill_id as move_skill_id
exclude = exclude_keys or set() exclude = exclude_keys or set()
recent = recent_keys or set() recent = expand_skill_recent_keys(recent_keys)
seen: set[str] = set() exclude_sources = {_source_from_skill_key(k) for k in exclude if k}
flat: list[dict[str, Any]] = [] exclude_sources.update(k for k in exclude if k)
def _blocked(item: dict[str, Any]) -> bool:
primary = _skill_primary_id(item)
source = str(item.get("source") or "").strip()
if primary and primary in recent:
return True
if source and source in recent:
return True
if primary and primary in exclude:
return True
if source and (source in exclude_sources or source in exclude):
return True
return False
groups: list[dict[str, Any]] = []
seen_sources: set[str] = set()
move_rows: list[dict[str, Any]] = []
seen_move_ids: set[str] = set()
for move in moves: for move in moves:
key = move_skill_id(move) key = move_skill_id(move)
if not key or key in seen or key in exclude: if not key or key in seen_move_ids:
continue continue
seen.add(key) move_source = str(move.get("source") or "").strip()
flat.append(_move_to_skill_row(move)) if key in exclude or (move_source and move_source in exclude_sources):
pad_pool = max(limit * 5, len(flat), 50) continue
for item in full_items: if key in recent or (move_source and move_source in recent):
if len(flat) >= pad_pool: continue
seen_move_ids.add(key)
move_rows.append(_move_to_skill_row(move))
for group in _normalize_skill_source_groups(move_rows):
source = str(group.get("source") or "?")
if source in seen_sources or _blocked(group):
continue
seen_sources.add(source)
groups.append(group)
if len(groups) >= limit:
break break
for row in _flatten_skill_board_item(item):
key = board_skill_id(row) if len(groups) < limit:
if not key or key in seen or key in exclude or key in recent: for group in _normalize_skill_source_groups(full_items):
if len(groups) >= limit:
break
source = str(group.get("source") or "?")
if source in seen_sources or _blocked(group):
continue continue
seen.add(key) seen_sources.add(source)
flat.append(row) groups.append(group)
return _prepare_grouped_wecom_skills(flat, limit=limit)
if not groups:
return [], set()
prepared = finalize_wecom_skill_groups(groups[:limit])
wecom_items = [_grouped_skill_to_wecom_item(x) for x in prepared]
# 供同日 Hot 排除：主键 + source
keys: set[str] = set()
for item in groups[:limit]:
primary = _skill_primary_id(item)
if primary:
keys.add(primary)
source = str(item.get("source") or "").strip()
if source:
keys.add(source)
return wecom_items[:limit], keys
def build_skills_delta_sections( def build_skills_delta_sections(
@@ -508,8 +585,8 @@ def build_skills_delta_sections(
*, *,
trending_full: list[dict[str, Any]] | None = None, trending_full: list[dict[str, Any]] | None = None,
hot_full: list[dict[str, Any]] | None = None, hot_full: list[dict[str, Any]] | None = None,
trending_limit: int = 10, trending_limit: int = 5,
hot_limit: int = 10, hot_limit: int = 5,
pad: bool = False, pad: bool = False,
recent_trending: set[str] | None = None, recent_trending: set[str] | None = None,
recent_hot: set[str] | None = None, recent_hot: set[str] | None = None,
@@ -517,11 +594,14 @@ def build_skills_delta_sections(
sections: list[str] = [] sections: list[str] = []
trending_keys: set[str] = set() trending_keys: set[str] = set()
if pad: if pad:
skill_recent = expand_skill_recent_keys(
(recent_trending or set()) | (recent_hot or set())
)
t_items, trending_keys = _merge_skill_board_items( t_items, trending_keys = _merge_skill_board_items(
trending_moves, trending_moves,
trending_full or [], trending_full or [],
trending_limit, trending_limit,
recent_keys=recent_trending, recent_keys=skill_recent,
) )
if t_items: if t_items:
lines = [f"{ICONS['trending']} **Skills Trending Top {len(t_items)}**"] lines = [f"{ICONS['trending']} **Skills Trending Top {len(t_items)}**"]
@@ -533,7 +613,7 @@ def build_skills_delta_sections(
hot_full or [], hot_full or [],
hot_limit, hot_limit,
exclude_keys=trending_keys, exclude_keys=trending_keys,
recent_keys=recent_hot, recent_keys=skill_recent,
) )
if h_items: if h_items:
lines = [f"{ICONS['hot']} **Skills Hot Top {len(h_items)}**"] lines = [f"{ICONS['hot']} **Skills Hot Top {len(h_items)}**"]
@@ -585,7 +665,7 @@ def _merge_github_board_items(
for move in moves: for move in moves:
repo = _github_move_to_repo(move) repo = _github_move_to_repo(move)
key = str(repo.get("repo") or "") key = str(repo.get("repo") or "")
if not key or key in seen: if not key or key in seen or key in recent:
continue continue
seen.add(key) seen.add(key)
merged.append(repo) merged.append(repo)
@@ -607,15 +687,20 @@ def build_github_delta_sections(
github_trending: list[dict[str, Any]] | None = None, github_trending: list[dict[str, Any]] | None = None,
github_emerging: list[dict[str, Any]] | None = None, github_emerging: list[dict[str, Any]] | None = None,
github_topic: list[dict[str, Any]] | None = None, github_topic: list[dict[str, Any]] | None = None,
trending_limit: int = 10, trending_limit: int = 5,
emerging_limit: int = 10, emerging_limit: int = 5,
topic_limit: int = 10, topic_limit: int = 5,
pad: bool = False, pad: bool = False,
recent_board_keys: dict[str, set[str]] | None = None, recent_board_keys: dict[str, set[str]] | None = None,
) -> str: ) -> str:
sections: list[str] = [] sections: list[str] = []
recent = recent_board_keys or {} recent = recent_board_keys or {}
if pad: if pad:
github_recent = (
(recent.get("github_trending") or set())
| (recent.get("github_emerging") or set())
| (recent.get("github_topic") or set())
)
mapping = [ mapping = [
("github_trending_moves", "github_trending", github_trending or [], trending_limit, "github", "GitHub Trending", False), ("github_trending_moves", "github_trending", github_trending or [], trending_limit, "github", "GitHub Trending", False),
("github_emerging_moves", "github_emerging", github_emerging or [], emerging_limit, "emerging", "GitHub 新兴", True), ("github_emerging_moves", "github_emerging", github_emerging or [], emerging_limit, "emerging", "GitHub 新兴", True),
@@ -626,8 +711,9 @@ def build_github_delta_sections(
movement.get(move_key) or [], movement.get(move_key) or [],
full_repos, full_repos,
limit, limit,
recent_repos=recent.get(board_key), recent_repos=github_recent,
) )
github_recent |= {str(r.get("repo") or "") for r in repos if r.get("repo")}
if not repos: if not repos:
continue continue
lines = [f"{ICONS[icon_key]} **{label} Top {len(repos)}**"] lines = [f"{ICONS[icon_key]} **{label} Top {len(repos)}**"]
@@ -678,11 +764,11 @@ def resolve_wecom_board_items(
github_trending: list[dict[str, Any]] | None = None, github_trending: list[dict[str, Any]] | None = None,
github_emerging: list[dict[str, Any]] | None = None, github_emerging: list[dict[str, Any]] | None = None,
github_topic: list[dict[str, Any]] | None = None, github_topic: list[dict[str, Any]] | None = None,
wecom_trending: int = 10, wecom_trending: int = 5,
wecom_hot: int = 10, wecom_hot: int = 5,
wecom_github: int = 10, wecom_github: int = 5,
wecom_emerging: int = 10, wecom_emerging: int = 5,
wecom_topic: int = 10, wecom_topic: int = 5,
pad: bool = False, pad: bool = False,
date_str: str | None = None, date_str: str | None = None,
trending_pad: list[dict[str, Any]] | None = None, trending_pad: list[dict[str, Any]] | None = None,
@@ -704,8 +790,21 @@ def resolve_wecom_board_items(
} }
recent_board_keys: dict[str, set[str]] = {} recent_board_keys: dict[str, set[str]] = {}
skill_recent: set[str] = set()
github_recent: set[str] = set()
if pad and date_str: if pad and date_str:
recent_board_keys = load_recent_board_keys(date_str) recent_board_keys = load_recent_board_keys(date_str)
# Trending / Hot 共用周去重：任一类出现过的 source 两边都不再展示
skill_recent = expand_skill_recent_keys(
(recent_board_keys.get("skills_trending") or set())
| (recent_board_keys.get("skills_hot") or set())
)
# GitHub 三榜共用周去重：任一类出现过的 repo 各榜都不再展示
github_recent = (
(recent_board_keys.get("github_trending") or set())
| (recent_board_keys.get("github_emerging") or set())
| (recent_board_keys.get("github_topic") or set())
)
t_moves, h_moves = partition_skill_moves_for_wecom( t_moves, h_moves = partition_skill_moves_for_wecom(
movement.get("skills_trending_moves") or [], movement.get("skills_trending_moves") or [],
@@ -716,36 +815,41 @@ def resolve_wecom_board_items(
t_moves, t_moves,
trending_pad if trending_pad else trending, trending_pad if trending_pad else trending,
wecom_trending, wecom_trending,
recent_keys=recent_board_keys.get("skills_trending"), recent_keys=skill_recent,
) )
h_items, _ = _merge_skill_board_items( h_items, _ = _merge_skill_board_items(
h_moves, h_moves,
hot_pad if hot_pad else hot, hot_pad if hot_pad else hot,
wecom_hot, wecom_hot,
exclude_keys=trending_keys, exclude_keys=trending_keys,
recent_keys=recent_board_keys.get("skills_hot"), recent_keys=skill_recent,
)
gt_items = _merge_github_board_items(
movement.get("github_trending_moves") or [],
github_trending_pad if github_trending_pad else (github_trending or []),
wecom_github,
recent_repos=github_recent,
)
github_recent |= {str(r.get("repo") or "") for r in gt_items if r.get("repo")}
ge_items = _merge_github_board_items(
movement.get("github_emerging_moves") or [],
github_emerging_pad if github_emerging_pad else (github_emerging or []),
wecom_emerging,
recent_repos=github_recent,
)
github_recent |= {str(r.get("repo") or "") for r in ge_items if r.get("repo")}
gtopic_items = _merge_github_board_items(
movement.get("github_topic_moves") or [],
github_topic_pad if github_topic_pad else (github_topic or []),
wecom_topic,
recent_repos=github_recent,
) )
return { return {
"skills_trending": t_items, "skills_trending": t_items,
"skills_hot": h_items, "skills_hot": h_items,
"github_trending": _merge_github_board_items( "github_trending": gt_items,
movement.get("github_trending_moves") or [], "github_emerging": ge_items,
github_trending_pad if github_trending_pad else (github_trending or []), "github_topic": gtopic_items,
wecom_github,
recent_repos=recent_board_keys.get("github_trending"),
),
"github_emerging": _merge_github_board_items(
movement.get("github_emerging_moves") or [],
github_emerging_pad if github_emerging_pad else (github_emerging or []),
wecom_emerging,
recent_repos=recent_board_keys.get("github_emerging"),
),
"github_topic": _merge_github_board_items(
movement.get("github_topic_moves") or [],
github_topic_pad if github_topic_pad else (github_topic or []),
wecom_topic,
recent_repos=recent_board_keys.get("github_topic"),
),
} }
t_flat = [_move_to_skill_row(m) for m in t_moves] t_flat = [_move_to_skill_row(m) for m in t_moves]
@@ -778,11 +882,11 @@ def replace_wecom_board_sections(
github_trending: list[dict[str, Any]] | None = None, github_trending: list[dict[str, Any]] | None = None,
github_emerging: list[dict[str, Any]] | None = None, github_emerging: list[dict[str, Any]] | None = None,
github_topic: list[dict[str, Any]] | None = None, github_topic: list[dict[str, Any]] | None = None,
wecom_trending: int = 10, wecom_trending: int = 5,
wecom_hot: int = 10, wecom_hot: int = 5,
wecom_github: int = 10, wecom_github: int = 5,
wecom_emerging: int = 10, wecom_emerging: int = 5,
wecom_topic: int = 10, wecom_topic: int = 5,
pad: bool = False, pad: bool = False,
date_str: str | None = None, date_str: str | None = None,
trending_pad: list[dict[str, Any]] | None = None, trending_pad: list[dict[str, Any]] | None = None,
@@ -854,11 +958,11 @@ def replace_wecom_skill_sections(
github_trending: list[dict[str, Any]] | None = None, github_trending: list[dict[str, Any]] | None = None,
github_emerging: list[dict[str, Any]] | None = None, github_emerging: list[dict[str, Any]] | None = None,
github_topic: list[dict[str, Any]] | None = None, github_topic: list[dict[str, Any]] | None = None,
wecom_trending: int = 10, wecom_trending: int = 5,
wecom_hot: int = 10, wecom_hot: int = 5,
wecom_github: int = 10, wecom_github: int = 5,
wecom_emerging: int = 10, wecom_emerging: int = 5,
wecom_topic: int = 10, wecom_topic: int = 5,
pad: bool = False, pad: bool = False,
date_str: str | None = None, date_str: str | None = None,
trending_pad: list[dict[str, Any]] | None = None, trending_pad: list[dict[str, Any]] | None = None,

65
daily/generate.py
View File

@@ -510,23 +510,26 @@ def _format_skill_section(items: list[dict[str, Any]], *, hot: bool = False) ->
def generate_report() -> tuple[str, str, Path, Path]: def generate_report() -> tuple[str, str, Path, Path]:
trending_n = env_int("DAILY_TRENDING_LIMIT", 150) # Hot/Trending 前排同 source 极密，需更深抓取才能凑够展示用的唯一 source
hot_n = max(env_int("DAILY_HOT_LIMIT", 150), compare_depth()) trending_n = env_int("DAILY_TRENDING_LIMIT", 400)
hot_n = max(env_int("DAILY_HOT_LIMIT", 400), compare_depth())
compare_n = compare_depth() compare_n = compare_depth()
skill_pool = max(10, env_int("DAILY_WECOM_SKILL_POOL", 200)) skill_pool = max(10, env_int("DAILY_WECOM_SKILL_POOL", 400))
wecom_trending = env_int("DAILY_WECOM_TRENDING", 10) wecom_trending = env_int("DAILY_WECOM_TRENDING", 5)
wecom_hot = env_int("DAILY_WECOM_HOT", 10) wecom_hot = env_int("DAILY_WECOM_HOT", 5)
pad_pool = wecom_pad_pool_size(max(wecom_trending, wecom_hot, 10)) pad_pool = wecom_pad_pool_size(max(wecom_trending, wecom_hot, 5))
skill_pool = max(skill_pool, pad_pool) skill_pool = max(skill_pool, pad_pool)
github_limit = env_int("DAILY_GITHUB_TRENDING_LIMIT", 10) github_limit = env_int("DAILY_GITHUB_TRENDING_LIMIT", 10)
wecom_github = env_int("DAILY_WECOM_GITHUB_TRENDING", env_int("DAILY_WECOM_REPOS", 10)) wecom_github = env_int("DAILY_WECOM_GITHUB_TRENDING", env_int("DAILY_WECOM_REPOS", 5))
github_fetch_n = max(github_limit, compare_n, wecom_github, pad_pool) # 周去重后顶刊 sticky，HTML/~30 条不够补满；深池默认 100（Search 已分页）
github_pool = max(pad_pool, env_int("DAILY_GITHUB_POOL", 100))
github_fetch_n = max(github_limit, compare_n, wecom_github, github_pool)
emerging_limit = env_int("DAILY_GITHUB_EMERGING_LIMIT", 10) emerging_limit = env_int("DAILY_GITHUB_EMERGING_LIMIT", 10)
wecom_emerging = env_int("DAILY_WECOM_GITHUB_EMERGING", 10) wecom_emerging = env_int("DAILY_WECOM_GITHUB_EMERGING", 5)
emerging_fetch_n = max(emerging_limit, compare_n, wecom_emerging, pad_pool) emerging_fetch_n = max(emerging_limit, compare_n, wecom_emerging, github_pool)
topic_limit = env_int("DAILY_GITHUB_TOPIC_LIMIT", 10) topic_limit = env_int("DAILY_GITHUB_TOPIC_LIMIT", 10)
wecom_topic = env_int("DAILY_WECOM_GITHUB_TOPIC", 10) wecom_topic = env_int("DAILY_WECOM_GITHUB_TOPIC", 5)
topic_fetch_n = max(topic_limit, compare_n, wecom_topic, pad_pool) topic_fetch_n = max(topic_limit, compare_n, wecom_topic, github_pool)
feed = load_feed(force=True) feed = load_feed(force=True)
prev_ids = _load_snapshot() prev_ids = _load_snapshot()
@@ -576,10 +579,21 @@ def generate_report() -> tuple[str, str, Path, Path]:
} }
pool = max(board_pool_size(), skill_pool, pad_pool) pool = max(board_pool_size(), skill_pool, pad_pool)
recent_shown = load_recent_shown_keys(date_str) recent_shown = load_recent_shown_keys(date_str)
from daily.skills_group import expand_skill_recent_keys
skill_recent = expand_skill_recent_keys(
recent_shown["skills_trending"] | recent_shown["skills_hot"]
)
# GitHub 三榜共用周去重：任一类出现过的 repo 各榜都不再展示
github_recent = (
recent_shown["github_trending"]
| recent_shown["github_emerging"]
| recent_shown["github_topic"]
)
selected_trending = board_select( selected_trending = board_select(
board="skills_trending", board="skills_trending",
items=trending, items=trending,
recent_keys=recent_shown["skills_trending"], recent_keys=skill_recent,
limit=wecom_trending, limit=wecom_trending,
pool_size=pool, pool_size=pool,
kind="skill", kind="skill",
@@ -587,7 +601,7 @@ def generate_report() -> tuple[str, str, Path, Path]:
selected_hot = board_select( selected_hot = board_select(
board="skills_hot", board="skills_hot",
items=hot, items=hot,
recent_keys=recent_shown["skills_hot"], recent_keys=skill_recent,
limit=wecom_hot, limit=wecom_hot,
pool_size=pool, pool_size=pool,
kind="skill", kind="skill",
@@ -595,23 +609,25 @@ def generate_report() -> tuple[str, str, Path, Path]:
selected_github = board_select( selected_github = board_select(
board="github_trending", board="github_trending",
items=github_trending, items=github_trending,
recent_keys=recent_shown["github_trending"], recent_keys=github_recent,
limit=wecom_github, limit=wecom_github,
pool_size=pool, pool_size=pool,
kind="github", kind="github",
) )
github_recent |= {str(r.get("repo") or "") for r in selected_github if r.get("repo")}
selected_emerging = board_select( selected_emerging = board_select(
board="github_emerging", board="github_emerging",
items=github_emerging, items=github_emerging,
recent_keys=recent_shown["github_emerging"], recent_keys=github_recent,
limit=wecom_emerging, limit=wecom_emerging,
pool_size=pool, pool_size=pool,
kind="github", kind="github",
) )
github_recent |= {str(r.get("repo") or "") for r in selected_emerging if r.get("repo")}
selected_topic = board_select( selected_topic = board_select(
board="github_topic", board="github_topic",
items=github_topic, items=github_topic,
recent_keys=recent_shown["github_topic"], recent_keys=github_recent,
limit=wecom_topic, limit=wecom_topic,
pool_size=pool, pool_size=pool,
kind="github", kind="github",
@@ -842,7 +858,7 @@ def generate_report() -> tuple[str, str, Path, Path]:
gt_pad = board_select( gt_pad = board_select(
board="skills_trending", board="skills_trending",
items=trending, items=trending,
recent_keys=recent_shown["skills_trending"], recent_keys=skill_recent,
limit=pad_pool, limit=pad_pool,
pool_size=pool, pool_size=pool,
kind="skill", kind="skill",
@@ -850,7 +866,7 @@ def generate_report() -> tuple[str, str, Path, Path]:
gh_pad = board_select( gh_pad = board_select(
board="skills_hot", board="skills_hot",
items=hot, items=hot,
recent_keys=recent_shown["skills_hot"], recent_keys=skill_recent,
limit=pad_pool, limit=pad_pool,
pool_size=pool, pool_size=pool,
kind="skill", kind="skill",
@@ -858,12 +874,17 @@ def generate_report() -> tuple[str, str, Path, Path]:
wecom_github_items = [_prepare_github_item(item) for item in selected_github] wecom_github_items = [_prepare_github_item(item) for item in selected_github]
wecom_emerging_items = [_prepare_github_item(item) for item in selected_emerging] wecom_emerging_items = [_prepare_github_item(item) for item in selected_emerging]
wecom_topic_items = [_prepare_github_item(item) for item in selected_topic] wecom_topic_items = [_prepare_github_item(item) for item in selected_topic]
github_pad_recent = (
recent_shown["github_trending"]
| recent_shown["github_emerging"]
| recent_shown["github_topic"]
)
wecom_github_pad = [ wecom_github_pad = [
_prepare_github_item(item) _prepare_github_item(item)
for item in board_select( for item in board_select(
board="github_trending", board="github_trending",
items=github_trending, items=github_trending,
recent_keys=recent_shown["github_trending"], recent_keys=github_pad_recent,
limit=pad_pool, limit=pad_pool,
pool_size=pool, pool_size=pool,
kind="github", kind="github",
@@ -874,7 +895,7 @@ def generate_report() -> tuple[str, str, Path, Path]:
for item in board_select( for item in board_select(
board="github_emerging", board="github_emerging",
items=github_emerging, items=github_emerging,
recent_keys=recent_shown["github_emerging"], recent_keys=github_pad_recent,
limit=pad_pool, limit=pad_pool,
pool_size=pool, pool_size=pool,
kind="github", kind="github",
@@ -885,7 +906,7 @@ def generate_report() -> tuple[str, str, Path, Path]:
for item in board_select( for item in board_select(
board="github_topic", board="github_topic",
items=github_topic, items=github_topic,
recent_keys=recent_shown["github_topic"], recent_keys=github_pad_recent,
limit=pad_pool, limit=pad_pool,
pool_size=pool, pool_size=pool,
kind="github", kind="github",

67
daily/github/search.py
View File

@@ -43,38 +43,59 @@ def search_github_repos(
logger.warning("GitHub Search 需要 GITHUB_TOKEN: %s", query[:80]) logger.warning("GitHub Search 需要 GITHUB_TOKEN: %s", query[:80])
return [] return []
target = max(1, min(int(limit), 1000))
per_page = min(100, target)
repos: list[dict[str, Any]] = []
seen: set[str] = set()
page = 1
try: try:
with httpx.Client( with httpx.Client(
timeout=20.0, timeout=20.0,
verify=certifi.where(), verify=certifi.where(),
headers=github_api_headers(), headers=github_api_headers(),
) as client: ) as client:
resp = client.get( while len(repos) < target:
"https://api.github.com/search/repositories", resp = client.get(
params={ "https://api.github.com/search/repositories",
"q": query, params={
"sort": sort, "q": query,
"order": "desc", "sort": sort,
"per_page": min(max(limit, 1), 30), "order": "desc",
}, "per_page": per_page,
) "page": page,
if resp.status_code != 200: },
logger.warning("GitHub Search 失败 (%s): %s", resp.status_code, query[:80]) )
return [] if resp.status_code != 200:
items = resp.json().get("items") or [] logger.warning(
"GitHub Search 失败 (%s page=%s): %s",
resp.status_code,
page,
query[:80],
)
break
items = resp.json().get("items") or []
if not items:
break
for item in items:
full_name = item.get("full_name") or ""
if not full_name or full_name in seen:
continue
seen.add(full_name)
repos.append(_repo_from_api_item(item, source="api-search"))
if len(repos) >= target:
break
if len(items) < per_page:
break
page += 1
# Search API 最多约 1000 条 / 10 页
if page > 10:
break
except Exception as exc: except Exception as exc:
logger.warning("GitHub Search 异常: %s", exc) logger.warning("GitHub Search 异常: %s", exc)
return [] return repos[:target]
repos: list[dict[str, Any]] = [] return repos[:target]
for item in items:
full_name = item.get("full_name") or ""
if not full_name:
continue
repos.append(_repo_from_api_item(item, source="api-search"))
if len(repos) >= limit:
break
return repos
def _date_days_ago(days: int) -> str: def _date_days_ago(days: int) -> str:

20
daily/llm_client.py
View File

@@ -76,16 +76,9 @@ def _cursor_chat(system: str, user: str) -> str:
return "" return ""
from cursor_sdk import Agent, AgentOptions, CursorAgentError, LocalAgentOptions from cursor_sdk import Agent, AgentOptions, CursorAgentError, LocalAgentOptions
from daily.config import ensure_bot_on_path from daily.bridge_manager import warm_cursor_bridge
ensure_bot_on_path()
try:
from bridge_manager import warm_cursor_bridge
except ImportError:
warm_cursor_bridge = lambda: None # noqa: E731
cwd = env("DAILY_CURSOR_CWD") or str(ROOT) cwd = env("DAILY_CURSOR_CWD") or str(ROOT)
# bridge_manager 读 bot env_config 的 CURSOR_CWD，早报侧须先对齐工作目录
os.environ["CURSOR_CWD"] = cwd os.environ["CURSOR_CWD"] = cwd
warm_cursor_bridge() warm_cursor_bridge()
model = env("CURSOR_MODEL") or "composer-2.5" model = env("CURSOR_MODEL") or "composer-2.5"
@@ -115,5 +108,16 @@ def llm_chat(system: str, user: str) -> str:
return "" return ""
def has_cursor_configured() -> bool:
return bool((env("CURSOR_API_KEY") or "").strip())
def cursor_agent_prompt(system: str, user: str) -> str:
"""仅 Cursor SDK Agent（可用 WebSearch 等工具），不走 OpenAI 兼容 API。"""
if not has_cursor_configured():
return ""
return _cursor_chat(system, user)
def has_llm_configured() -> bool: def has_llm_configured() -> bool:
return bool(env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY") or env("CURSOR_API_KEY")) return bool(env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY") or env("CURSOR_API_KEY"))

13
daily/news/__init__.py
View File

@@ -1,13 +1,26 @@
from daily.news.fetch import ( from daily.news.fetch import (
fetch_ai_news, fetch_ai_news,
fetch_cn_ai_news, fetch_cn_ai_news,
format_cn_news_section, format_cn_news_section,
format_news_section, format_news_section,
) )
__all__ = [ __all__ = [
"fetch_ai_news", "fetch_ai_news",
"fetch_cn_ai_news", "fetch_cn_ai_news",
"format_news_section", "format_news_section",
"format_cn_news_section", "format_cn_news_section",
] ]

2
daily/news/feeds.py
View File

@@ -27,7 +27,7 @@ NEWS_CATEGORIES: tuple[NewsCategory, ...] = (
name="厂商官方", name="厂商官方",
icon="🏢", icon="🏢",
feeds=( feeds=(
NewsFeed("Anthropic Claude 更新", "https://docs.anthropic.com/en/release-notes/feed"), NewsFeed("Anthropic Claude 更新", "https://platform.claude.com/docs/en/release-notes/overview"),
NewsFeed("OpenAI", "https://openai.com/news/rss.xml"), NewsFeed("OpenAI", "https://openai.com/news/rss.xml"),
NewsFeed("Google AI", "https://blog.google/technology/ai/rss/"), NewsFeed("Google AI", "https://blog.google/technology/ai/rss/"),
NewsFeed("DeepMind", "https://deepmind.google/blog/rss.xml"), NewsFeed("DeepMind", "https://deepmind.google/blog/rss.xml"),

9
daily/news/feeds_cn.py
View File

@@ -44,7 +44,6 @@ CN_NEWS_CATEGORIES: tuple[NewsCategory, ...] = (
icon="📰", icon="📰",
feeds=( feeds=(
NewsFeed("量子位", "https://www.qbitai.com/feed"), NewsFeed("量子位", "https://www.qbitai.com/feed"),
NewsFeed("InfoQ 中文", "https://www.infoq.cn/feed/AI"),
), ),
), ),
NewsCategory( NewsCategory(
@@ -60,12 +59,4 @@ CN_NEWS_CATEGORIES: tuple[NewsCategory, ...] = (
), ),
), ),
), ),
NewsCategory(
id="dev",
name="开发者社区",
icon="💻",
feeds=(
NewsFeed("掘金", "https://juejin.cn/rss", ai_filter=True),
),
),
) )

2
daily/report_data.py
View File

@@ -94,7 +94,7 @@ def _slim_news_items(
def _wecom_skill_pool() -> int: def _wecom_skill_pool() -> int:
return max(10, env_int("DAILY_WECOM_SKILL_POOL", 200)) return max(10, env_int("DAILY_WECOM_SKILL_POOL", 400))
def build_llm_input( def build_llm_input(

284
daily/scheduler.py Normal file
View File

@@ -0,0 +1,284 @@
"""常驻调度：按配置时刻生成早报并推送企微。"""
from __future__ import annotations
import json
import logging
import subprocess
import sys
import time
from dataclasses import dataclass
from datetime import date, datetime, time as dt_time, timedelta
from pathlib import Path
from zoneinfo import ZoneInfo
from daily.config import (
CACHE_DIR,
LOG_DIR,
OUTPUT_DIR,
ROOT,
schedule_generate_at,
schedule_push_at,
schedule_timezone_name,
)
logger = logging.getLogger(__name__)
_STATE_FILE = CACHE_DIR / "scheduler-state.json"
_POLL_SECONDS = 15
@dataclass(frozen=True)
class ClockTime:
hour: int
minute: int
@dataclass
class SchedulerState:
last_generate_date: str | None = None
last_push_date: str | None = None
@classmethod
def load(cls) -> SchedulerState:
if not _STATE_FILE.exists():
return cls()
try:
raw = json.loads(_STATE_FILE.read_text(encoding="utf-8"))
except (OSError, ValueError):
return cls()
if not isinstance(raw, dict):
return cls()
return cls(
last_generate_date=raw.get("last_generate_date"),
last_push_date=raw.get("last_push_date"),
)
def save(self) -> None:
_STATE_FILE.parent.mkdir(parents=True, exist_ok=True)
payload = {
"last_generate_date": self.last_generate_date,
"last_push_date": self.last_push_date,
}
_STATE_FILE.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
def parse_hhmm(value: str) -> ClockTime:
raw = (value or "").strip()
parts = raw.split(":", 1)
if len(parts) != 2:
raise ValueError(f"无效时间格式: {value!r}，应为 HH:MM")
hour = int(parts[0])
minute = int(parts[1])
if not (0 <= hour <= 23 and 0 <= minute <= 59):
raise ValueError(f"无效时间: {value!r}")
return ClockTime(hour=hour, minute=minute)
def load_timezone() -> ZoneInfo:
name = schedule_timezone_name()
try:
return ZoneInfo(name)
except Exception as exc:
raise RuntimeError(f"无效时区 DAILY_SCHEDULE_TZ={name!r}") from exc
def _localize(day: date, clock: ClockTime, tz: ZoneInfo) -> datetime:
return datetime.combine(day, dt_time(clock.hour, clock.minute), tz)
def next_occurrence_after(clock: ClockTime, tz: ZoneInfo, after: datetime) -> datetime:
local = after.astimezone(tz)
candidate = local.replace(hour=clock.hour, minute=clock.minute, second=0, microsecond=0)
if candidate <= local:
candidate += timedelta(days=1)
return candidate
def _today_slot(day: date, clock: ClockTime, tz: ZoneInfo) -> datetime:
return _localize(day, clock, tz)
def _run_daily_subcommand(subcmd: str, *extra: str) -> int:
cmd = [sys.executable, "-m", "daily", subcmd, *extra]
logger.info("执行: %s", " ".join(cmd))
proc = subprocess.run(cmd, cwd=str(ROOT), check=False)
return int(proc.returncode)
def run_generate() -> int:
return _run_daily_subcommand("generate")
def run_push_for_date(date_str: str) -> int:
report = OUTPUT_DIR / f"{date_str}.wecom.md"
if not report.exists():
logger.error("推送失败：报告不存在 %s", report)
return 1
return _run_daily_subcommand("push", str(report))
def _setup_logging() -> Path:
LOG_DIR.mkdir(parents=True, exist_ok=True)
log_path = LOG_DIR / "scheduler.log"
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(message)s",
handlers=[
logging.FileHandler(log_path, encoding="utf-8"),
logging.StreamHandler(sys.stdout),
],
)
return log_path
def _sleep_until(target: datetime, tz: ZoneInfo) -> None:
while True:
now = datetime.now(tz)
seconds = (target - now).total_seconds()
if seconds <= 0:
return
time.sleep(min(seconds, _POLL_SECONDS))
def plan_next_action(
*,
now: datetime,
tz: ZoneInfo,
state: SchedulerState,
generate_at: ClockTime,
push_at: ClockTime,
) -> tuple[datetime, str] | None:
"""返回下一次应执行的动作；若今日已全部完成则返回明日 generate。"""
today = now.astimezone(tz).date()
today_str = today.isoformat()
gen_done = state.last_generate_date == today_str
push_done = state.last_push_date == today_str
gen_slot = _today_slot(today, generate_at, tz)
push_slot = _today_slot(today, push_at, tz)
# 推送窗口内：generate 未做则立即补跑（须先于 push）
if not gen_done and gen_slot <= now <= push_slot:
return now, "generate"
# 已过推送时刻：仅当 generate 已完成时补跑 push
if not push_done and gen_done and now >= push_slot:
return now, "push"
candidates: list[tuple[datetime, str]] = []
if not gen_done and gen_slot > now:
candidates.append((gen_slot, "generate"))
if not push_done and push_slot > now:
candidates.append((push_slot, "push"))
if candidates:
return min(candidates, key=lambda item: item[0])
tomorrow_gen = next_occurrence_after(generate_at, tz, now)
return tomorrow_gen, "generate"
def run_scheduled_action(action: str, *, today_str: str) -> int:
if action == "generate":
return run_generate()
if action == "push":
return run_push_for_date(today_str)
raise ValueError(f"未知动作: {action}")
def tick_once(
*,
now: datetime | None = None,
tz: ZoneInfo | None = None,
state: SchedulerState | None = None,
generate_at: ClockTime | None = None,
push_at: ClockTime | None = None,
dry_run: bool = False,
) -> SchedulerState:
tz = tz or load_timezone()
now = now or datetime.now(tz)
state = state or SchedulerState.load()
generate_at = generate_at or parse_hhmm(schedule_generate_at())
push_at = push_at or parse_hhmm(schedule_push_at())
today_str = now.astimezone(tz).date().isoformat()
planned = plan_next_action(
now=now,
tz=tz,
state=state,
generate_at=generate_at,
push_at=push_at,
)
if not planned:
return state
run_at, action = planned
if run_at > now:
if not dry_run:
logger.info("下次 %s @ %s (%s)", action, run_at.isoformat(), tz.key)
_sleep_until(run_at, tz)
elif not dry_run:
slot = _today_slot(now.astimezone(tz).date(), generate_at if action == "generate" else push_at, tz)
logger.info(
"补跑 %s（计划 %02d:%02d，当前 %s）",
action,
slot.hour,
slot.minute,
now.astimezone(tz).strftime("%H:%M"),
)
if dry_run:
logger.info("[dry-run] 将执行 %s @ %s", action, run_at.isoformat())
return state
logger.info("开始 %s（%s）", action, today_str)
code = run_scheduled_action(action, today_str=today_str)
if code != 0:
logger.error("%s 失败，exit=%s", action, code)
else:
if action == "generate":
state.last_generate_date = today_str
elif action == "push":
state.last_push_date = today_str
state.save()
logger.info("%s 完成", action)
return state
def main() -> int:
dry_run = "--dry-run" in sys.argv[1:]
once = "--once" in sys.argv[1:]
log_path = _setup_logging()
tz = load_timezone()
generate_at = parse_hhmm(schedule_generate_at())
push_at = parse_hhmm(schedule_push_at())
logger.info(
"调度器启动 tz=%s generate=%02d:%02d push=%02d:%02d log=%s",
tz.key,
generate_at.hour,
generate_at.minute,
push_at.hour,
push_at.minute,
log_path,
)
state = SchedulerState.load()
try:
while True:
state = tick_once(
tz=tz,
state=state,
generate_at=generate_at,
push_at=push_at,
dry_run=dry_run,
)
if once or dry_run:
break
except KeyboardInterrupt:
logger.info("调度器已停止（KeyboardInterrupt）")
return 0
return 0
if __name__ == "__main__":
raise SystemExit(main())

79
daily/skills_board.py
View File

@@ -2,14 +2,16 @@
from __future__ import annotations from __future__ import annotations
import json
import logging import logging
import re import re
import time
from typing import Any, Literal from typing import Any, Literal
import certifi import certifi
import httpx import httpx
from daily.config import env from daily.config import CACHE_DIR, env
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -17,6 +19,81 @@ Board = Literal["trending", "hot"]
SKILLS_SITE = "https://www.skills.sh" SKILLS_SITE = "https://www.skills.sh"
USER_AGENT = "Mozilla/5.0 (compatible; skills-hot-daily/1.0; +https://skills.sh)" USER_AGENT = "Mozilla/5.0 (compatible; skills-hot-daily/1.0; +https://skills.sh)"
FEED_URLS = [
"https://cdn.jsdelivr.net/gh/NeverSight/skills.sh_feed@main/data/feed.json",
"https://raw.githubusercontent.com/NeverSight/skills.sh_feed/main/data/feed.json",
]
FEED_CACHE_TTL = 600
FEED_CACHE_FILE = CACHE_DIR / "feed.json"
_feed_cache: dict[str, Any] = {"data": None, "fetched_at": 0.0}
def format_installs(n: int | float) -> str:
if n >= 1_000_000:
return f"{n / 1_000_000:.1f}M"
if n >= 1_000:
return f"{n / 1_000:.1f}K"
return str(int(n))
def _fetch_feed_json(url: str) -> dict[str, Any]:
headers = {"User-Agent": USER_AGENT, "Accept": "application/json"}
with httpx.Client(
timeout=httpx.Timeout(20.0, connect=10.0),
verify=certifi.where(),
follow_redirects=True,
) as client:
resp = client.get(url, headers=headers)
resp.raise_for_status()
return resp.json()
def _load_feed_disk_cache() -> dict[str, Any] | None:
if not FEED_CACHE_FILE.exists():
return None
try:
return json.loads(FEED_CACHE_FILE.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
logger.warning("读取 feed 本地缓存失败: %s", exc)
return None
def _save_feed_disk_cache(data: dict[str, Any]) -> None:
FEED_CACHE_FILE.parent.mkdir(parents=True, exist_ok=True)
FEED_CACHE_FILE.write_text(json.dumps(data, ensure_ascii=False), encoding="utf-8")
def load_feed(force: bool = False) -> dict[str, Any]:
now = time.time()
if not force and _feed_cache["data"] and now - _feed_cache["fetched_at"] < FEED_CACHE_TTL:
return _feed_cache["data"]
errors: list[str] = []
for url in FEED_URLS:
for attempt in range(3):
try:
data = _fetch_feed_json(url)
_feed_cache["data"] = data
_feed_cache["fetched_at"] = now
_save_feed_disk_cache(data)
logger.info("skills feed 已更新: %s", url)
return data
except Exception as exc:
msg = f"{url} (#{attempt + 1}): {exc}"
errors.append(msg)
logger.debug("拉取失败 %s", msg)
time.sleep(0.5 * (attempt + 1))
stale = _load_feed_disk_cache()
if stale:
logger.warning("网络不可用，回退到 feed 本地缓存")
_feed_cache["data"] = stale
_feed_cache["fetched_at"] = now
return stale
raise RuntimeError(f"无法获取 skills 数据。最近错误: {errors[-1] if errors else 'unknown'}")
_SKILL_RE = re.compile( _SKILL_RE = re.compile(
r'\{"source":"(?P<source>[^"]+)","skillId":"(?P<skill_id>[^"]+)",' r'\{"source":"(?P<source>[^"]+)","skillId":"(?P<skill_id>[^"]+)",'
r'"name":"(?P<name>[^"]+)","installs":(?P<installs>\d+)' r'"name":"(?P<name>[^"]+)","installs":(?P<installs>\d+)'

22
daily/skills_group.py
View File

@@ -9,6 +9,28 @@ def skill_id(item: dict[str, Any]) -> str:
return str(item.get("id") or f"{item.get('source')}/{item.get('title')}") return str(item.get("id") or f"{item.get('source')}/{item.get('title')}")
def source_from_skill_key(key: str) -> str:
"""从 skill id（source/title…）还原 source：去掉最后一段 title。"""
parts = [p for p in str(key or "").split("/") if p]
if len(parts) >= 2:
return "/".join(parts[:-1])
return str(key or "").strip()
def expand_skill_recent_keys(keys: set[str] | None) -> set[str]:
"""周去重 blocklist：保留原始 key，并展开为 source。"""
out: set[str] = set()
for key in keys or set():
k = str(key or "").strip()
if not k:
continue
out.add(k)
src = source_from_skill_key(k)
if src:
out.add(src)
return out
def format_installs(n: int | float) -> str: def format_installs(n: int | float) -> str:
if n >= 1_000_000: if n >= 1_000_000:
return f"{n / 1_000_000:.1f}M" return f"{n / 1_000_000:.1f}M"

12
daily/text_utils.py
View File

@@ -15,7 +15,7 @@ def clip_text(text: str, limit: int) -> str:
def trim_brief(text: str, limit: int) -> str: def trim_brief(text: str, limit: int) -> str:
"""企微简要：控制在 limit 内，优先在句读处截断，不加省略号。""" """企微简要：控制在 limit 内，优先在句读/词边界截断，不加省略号。"""
text = _WS.sub(" ", (text or "").strip()) text = _WS.sub(" ", (text or "").strip())
if not text or limit <= 0 or len(text) <= limit: if not text or limit <= 0 or len(text) <= limit:
return text return text
@@ -27,4 +27,12 @@ def trim_brief(text: str, limit: int) -> str:
pos = text.find(sep) pos = text.find(sep)
if pos != -1 and pos + 1 <= limit: if pos != -1 and pos + 1 <= limit:
return text[: pos + 1] return text[: pos + 1]
return text[:limit].rstrip("，、；： ") for sep in (". ", "! ", "? ", "; "):
pos = text.rfind(sep, 0, limit + 1)
if pos != -1 and pos + 1 >= min(limit // 2, 20):
return text[: pos + 1].rstrip()
if len(text) > limit:
space = text.rfind(" ", 0, limit + 1)
if space >= min(limit // 2, 20):
return text[:space].rstrip("，、；： ,.;")
return text[:limit].rstrip("，、；： ,.;")

273
docs/design-wecom-delta-mode.md Normal file
View File

@@ -0,0 +1,273 @@
# Design: 企微早报 Delta 模式
Generated: 2026-07-09
Repo: daily-robots
Status: DRAFT
Mode: Builder
## Problem Statement
企微早报每天推送五榜 Top 10 + 18 条新闻，内容与前几日高度重复（`find-skills`、`openclaw`、飞书集群等长期霸榜）。读者真实需求是「今天有什么新变化」，而非「再读一遍黄页」。
根因：
1. `daily-agent/SKILL.md` 要求即使较昨日无新增，仍须完整列出 Top 榜。
2. `movement` 仅用于 opening / signals，列表区块仍全量渲染。
3. Trending 与 Hot 独立展示，同一 skill 描述写两遍。
4. 新闻 `DAILY_AI_NEWS_HOURS=72`，无已推送 link 去重，旧闻可连续出现。
## What Makes This Cool
把早报从「日报复印机」变成「变化通知」：只有新入榜、新新闻、编辑推荐时才占版面；榜全稳且无新新闻时静默不推。读者打开企微即知「今天值得扫一眼的是什么」。
## Explicit Non-Goals（已否决方案）
以下方案**不在本设计范围内**：
| 方案 | 状态 |
|------|------|
| 静态页 / 外链档案库 | ❌ 不做 |
| 今日一装（每天一个 `npx skills add`） | ❌ 不做 |
| 按星期轮换版面 | ❌ 不做 |
| 榜首锚点（稳定日仍展示 #1） | ❌ 不做 |
## Premises
1. 重复感主要来自**列表区块全量复印**，而非 opening 里引用榜首数字。
2. `output/*.data.json` 与 `daily/delta.py` 已具备新入榜对比能力，应上升为**列表渲染主数据源**。
3. 企微消息仍在应用内读完，不依赖外部页面。
4. 叙事层（opening、信号、首推、新闻）保持充实；缩短的是**榜单列表**，不是整报。
## Recommended Approach: Delta 模式
### 环境变量
```env
# full = 现有行为（全量 Top 榜列表）
# delta = 本设计（默认推荐）
DAILY_WECOM_MODE=delta
# 无对比基准时（首日或缺历史 data.json）是否自动 full 一次
DAILY_DELTA_BASELINE_FALLBACK=full # full | empty
# 推送闸门全不满足时是否跳过 webhook（仍写 output 文件）
DAILY_SKIP_PUSH_WHEN_SILENT=1
# 强制推送（忽略静默）
# DAILY_FORCE_PUSH=1
# 新闻：缩短窗口 + 去重天数
DAILY_AI_NEWS_HOURS=24
DAILY_NEWS_DEDUP_DAYS=7
```
### 推送闸门（Push Gate）
满足**任一**条件则生成并推送企微早报：
| 条件 | 数据源 |
|------|--------|
| 任榜单有新入条目 | `movement.*_moves` 非空 |
| 去重后仍有新新闻 | 国际或国内 AI 时讯 |
| 存在 `featured_pick` | Step 0 编辑推荐 |
| `DAILY_FORCE_PUSH=1` | 环境变量 |
**静默日**：以上皆不满足 → 不调用 webhook（`DAILY_SKIP_PUSH_WHEN_SILENT=1` 时）。
仍执行 `daily generate`，写入 `output/{date}.md`、`output/{date}.wecom.md`、`output/{date}.data.json` 留档。
**注意**：仅新闻有新、榜单全稳时**仍推送**，但 Skills/GitHub 列表区块整块省略（不是全天静默）。
### 列表渲染（Delta 列表）
`DAILY_WECOM_MODE=delta` 时：
#### Skills
- **仅展示** `movement.skills_trending_moves` / `movement.skills_hot_moves` 中的新入榜条目。
- **跨榜去重**：按 `skill_id`（`id` 或 `source/title`）合并；同一 skill 只出现一次，标注来源榜（如 `Trending #4 · Hot #2`）。
- **无新入**：该榜区块**整块不出现**（不写多行「较昨日无新增」）。
#### GitHub
- 仅展示 `movement.github_trending_moves`、`github_emerging_moves`、`github_topic_moves`。
- 无新入则区块省略。
#### 不包含
- 全量 Top N 列表
- 榜首锚点
- `（新入 … #n）` 括号标注（与现 `agent_workflow._strip_new_entry_notes` 一致，列表标题用 `[新入 #n]` 前缀即可）
### 固定骨架（不因 Delta 缩短）
Agent 模式（`DAILY_REPORT_MODE=agent`）下，以下区块**保持**：
- opening（2–3 句，首句含具体证据）
- headline / 今日主题
- 今日信号（3–5 条）
- 今日首推
- 国际 AI / 国内 AI 精选（条数仍由 `DAILY_WECOM_AI_NEWS` 等控制）
榜单变短；叙事与新闻不主动砍到 0。
### Full 模式逃生口
`DAILY_WECOM_MODE=full` 时行为与**当前生产一致**（`format_wecom.build_wecom_report` / `replace_wecom_skill_sections` 全量 Top N）。用于手动切回或对比测试。
### 首日 / 无历史基准
`find_previous_data(date)` 返回 `None` 时：
| `DAILY_DELTA_BASELINE_FALLBACK` | 行为 |
|----------------------------------|------|
| `full`（推荐） | 当日按 full 模式渲染列表一次；次日起 delta |
| `empty` | 当日列表区块为空；opening 须说明「首日报，暂无对比基准」 |
实现时在 `generate_report` 或 `build_llm_input` 传入 `baseline_date` 供 Agent 引用。
## News Dedup
### P0（本阶段）
- 维护 `cache/pushed-news-links.json`（或写入 `output/` 旁 cache）：最近 `DAILY_NEWS_DEDUP_DAYS` 天已出现在企微早报中的 `link` 集合。
- `prepare_wecom_news_items` / `prepare_wecom_cn_news_items` 输出前过滤已见 link。
- `DAILY_AI_NEWS_HOURS` 默认改为 `24`（`.env.example` 同步）。
### P1（可选后续）
- 标题归一化去重（同一事件多源报道）
- 每 `source_name` 每日上限 N 条
## Agent Skill 变更
文件：`skills/daily-agent/SKILL.md`
### 删除 / 修改
- 删除规则：「即使某榜较昨日无新增，仍须完整列出 Top 榜条目」。
- 删除：「禁止改用 movement 作为列表来源」（在 delta 模式下反转）。
### 新增
当 `DAILY_WECOM_MODE=delta`（或 llm_input 含 `wecom_mode: delta`）：
1. Agent **不写** Skills Trending / Hot / GitHub 列表（仍由 Python 插入，与现流程一致）。
2. opening / signals **可引用**榜首与 movement 摘要；禁止在 signals 重复列表已展示的同一事实。
3. 榜全稳时，signals 聚焦新闻与首推，不必编造榜单变化。
当 `wecom_mode: full` 时保持现有 SKILL 规则。
## Python 模块变更
| 模块 | 变更 |
|------|------|
| `daily/config.py` | `wecom_mode()`, `news_dedup_days()`, `skip_push_when_silent()`, `delta_baseline_fallback()` |
| `daily/delta.py` | 可选：`merge_skill_moves_for_wecom(trending_moves, hot_moves)` 跨榜去重 |
| `daily/format_wecom.py` | `build_skills_delta_section()`, `build_github_delta_section()`；`replace_wecom_skill_sections` 支持 delta |
| `daily/news/fetch.py` | `filter_pushed_news()` + cache 读写 |
| `daily/generate.py` | 推送闸门；baseline fallback；静默 skip push |
| `daily/report_data.py` | `llm_input` 增加 `wecom_mode`, `push_gate` 摘要 |
| `daily/agent_workflow.py` | 无逻辑变更；依赖 Python 插入 delta 列表 |
| `.env.example` | 新 env 文档 |
## 企微消息示例
### 有变化日
```markdown
📰 **早报 · 2026-07-10**
[opening：今天最大变化，含数字/条目名]
🎯 **{headline}**
💡 **今日信号**
> ...
📦 **今日首推**
[...]
🌍 **国际 AI · 精选 N**
...
📈 **Skills Trending 变化**
1. [新入 #4] [**xxx**](...) · ...
描述一行
🔥 **Skills Hot 变化**
1. [新入 #2] [**yyy**](...) · ...
🐙 **GitHub Trending 变化**
1. [新入 #4] [owner/repo](...) · ...
```
### 仅新闻有新（榜稳）
- 无 📈/🔥/🐙 区块
- opening 可一句：「榜单较昨日 Top15 无新入；以下为今日 AI 时讯。」
### 静默日
- 不推送企微
- `output/` 仍落盘；日志：`[silent] no push gate matched for 2026-07-10`
## Approaches Considered
### Approach A: 配置瘦身（缩 Top N、24h 新闻）
- Effort: S | Risk: Low
- 只减篇幅，榜头仍天天重复；未解决根因。
### Approach B: Delta 列表 + 推送闸门 + 新闻去重（本设计）
- Effort: M | Risk: Med
- 复用 `delta.py`；改 format + skill + push 逻辑。
### Approach C: 仅改 Agent 文案
- Effort: S | Risk: High
- Python 仍插入全量列表，规则冲突，不可持续。
**Recommendation: B** — 数据层与展示层一致，静默日与跨榜去重可测。
## Success Criteria
1. 连续 3 天对比 `output/*.data.json`：企微列表区块**重复 skill_id 占比**显著下降。
2. 榜全稳且无新新闻日：`DAILY_SKIP_PUSH_WHEN_SILENT=1` 时不发 webhook。
3. Trending/Hot 同一 skill 在列表中**最多出现 1 次**。
4. `DAILY_WECOM_MODE=full` 与现网行为一致（回归用）。
5. 首日 `DAILY_DELTA_BASELINE_FALLBACK=full` 不产生空列表投诉。
## Open Questions
1. 静默日是否需要在企微发一行「今日无更新」？当前设计：**不发**。
2. 新闻去重 cache 是否纳入 git？建议：**否**，放 `cache/`（已在 `.gitignore`）。
3. Classic 模式（非 agent）是否同步 delta？建议：**是**，同一 `format_wecom` 路径。
## Implementation Tasks
| ID | Priority | Task | Files |
|----|----------|------|-------|
| T1 | P1 | 新增 config helpers + `.env.example` | `daily/config.py`, `.env.example` |
| T2 | P1 | 新闻 link 去重 cache | `daily/news/fetch.py`, `daily/config.py` |
| T3 | P1 | Delta 列表渲染 + 跨榜去重 | `daily/format_wecom.py`, `daily/delta.py` |
| T4 | P1 | 推送闸门 + 静默 skip push | `daily/generate.py`, `daily/webhook.py` |
| T5 | P1 | baseline fallback full 一次 | `daily/generate.py` |
| T6 | P1 | 更新 `daily-agent/SKILL.md` | `skills/daily-agent/SKILL.md` |
| T7 | P2 | `llm_input` 传 `wecom_mode` / push 摘要 | `daily/report_data.py` |
| T8 | P2 | 单元测试：跨榜去重、推送闸门、新闻去重 | `tests/test_wecom_delta.py` |
## Test Plan
- [ ] 有 `2026-07-09.data.json` 时生成 `2026-07-10`：列表仅含新入项
- [ ] 人造「全稳 + 无新新闻」：不 push
- [ ] 人造「全稳 + 有新新闻」：push，无 Skills/GitHub 块
- [ ] `DAILY_WECOM_MODE=full` 输出与改前 `2026-07-09.wecom.md` 结构一致
- [ ] 无 baseline + `DAILY_DELTA_BASELINE_FALLBACK=full`：首日全量列表
- [ ] 同一 skill 在 Trending/Hot moves 均出现：列表只 1 条
## What I Noticed
- 重复感是**产品形态**问题，不是 Agent 文笔问题。
- 明确否决静态页、今日一装、轮换、锚点后，方案边界清晰，实现可分期。
- 推送闸门必须**把新闻算进去**，否则静默日会被新闻绕过。

1162
docs/superpowers/plans/2026-07-09-wecom-delta-mode.md Normal file
View File

File diff suppressed because it is too large Load Diff

604
docs/superpowers/plans/2026-07-14-wecom-diversity-dedup.md Normal file
View File

@@ -0,0 +1,604 @@
# 企微早报多样性与去重 Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** 实现企微早报硬去重多样性：五榜周去重（深池补满）、首推与昨日相同则改推（月去重）、叙事轴代码互斥、取消新闻「放宽窗口」凑数；且 `movement_baseline` 与 `wecom_shown_keys` 严格分离。
**Architecture:** 在 `daily generate` 管线加代码选择器：`board_select` 为 full/delta 唯一列表主人；周历史只读写 `data.wecom_shown_keys`（post-render）；`movement_baseline` 仍为 raw Top compare；`featured_resolve` 先定人再 research；`pick_narrative_axis` 代码选轴注入 Agent Step1；新闻关 backfill + 剥「放宽」前缀。
**Tech Stack:** Python 3.13+、现有 `unittest`/`pytest`、`daily/delta.py` / `format_wecom.py` / `featured_pick.py` / `news/fetch.py`、`output/*.data.json`
**Spec:** `docs/superpowers/specs/2026-07-14-wecom-diversity-dedup-design.md`（Status: APPROVED）
## Global Constraints
- `movement_baseline` **禁止**被展示历史覆写；周去重只读 `wecom_shown_keys`
- full/delta **唯一列表主人** = `board_select`（delta 的 pad 共用同一套 shown 历史）
- `DAILY_BOARD_DEDUP_DAYS` 默认 `7`；与 pad lookback 对齐且数据源同一
- `DAILY_FEATURED_DEDUP_DAYS` 默认 `30`
- `DAILY_NARRATIVE_AXIS_DAYS` 默认 `3`；轴枚举固定 7 个（见 Task 5）
- `DAILY_NEWS_BACKFILL` 默认 `0`（禁止旧闻凑数）
- research 补新闻年龄上限 = `DAILY_AI_NEWS_HOURS`（不得变相 48h 放宽）
- 不做：语义「同一类」、同日 Trending↔Hot 互斥、`FEATURED_FORCE`、关键短语硬匹配
- Commit 信息正文用中文（若本任务含 Commit 步）
---
## File Structure
| 文件 | 职责 |
|------|------|
| `daily/config.py` | 新 env：`board_dedup_days`、`board_pool_size`、`featured_dedup_days`、`theme_ban_days`、`narrative_axis_days`、`news_backfill_enabled` |
| `daily/board_history.py` | **新建** — `load_recent_shown_keys` / `extract_shown_keys` / `attach_wecom_shown_keys`（读写 `data.wecom_shown_keys`） |
| `daily/board_select.py` | **新建** — `board_select(...)` 周过滤+深池 |
| `daily/delta.py` | `load_recent_board_keys` 改为委托 `load_recent_shown_keys`（保留函数名兼容）；**不**改 `build_movement_baseline` |
| `daily/format_wecom.py` | pad 使用 shown keys；可选返回最终展示 items 供写回 |
| `daily/featured_pick.py` | `featured_identity_key`、`featured_resolve`、先定人再 research |
| `daily/narrative_axis.py` | **新建** — `NARRATIVE_AXES`、`pick_narrative_axis`、`load_recent_axes` |
| `daily/news/fetch.py` | `_apply_pushed_dedup_with_backfill` 尊重 `news_backfill_enabled()`；默认不塞回 |
| `daily/news/research.py` + `skills/daily-ai-news-research/SKILL.md` | 禁放宽文案；补入不超时窗 |
| `daily/text_utils.py` 或 `daily/news/sanitize.py` | `strip_news_relax_prefix(desc)` |
| `daily/agent_workflow.py` | Step1 注入 axis + 近 7 日 theme 软禁；强制覆写冲突轴 |
| `daily/generate.py` | 串联：select → featured → editorial → render → persist shown/key/axis |
| `daily/report_data.py` | data.json 可携 `wecom_shown_keys` / `featured_pick_key` / `narrative_axis`（写回可由 generate 合并） |
| `.env.example` | 文档化新变量 |
| `skills/daily-agent/SKILL.md` | `narrative_axis` 必填且等于输入指定轴 |
| `tests/test_board_select.py` | **新建** |
| `tests/test_board_history.py` | **新建** |
| `tests/test_featured_resolve.py` | **新建** |
| `tests/test_narrative_axis.py` | **新建** |
| `tests/test_news_relax.py` | **新建** |
| `tests/test_wecom_delta.py` | 回归：pad 不读 movement 当展示史 |
---
### Task 1: Config + shown-keys 历史层
**Files:**
- Modify: `daily/config.py`（文件末尾追加）
- Create: `daily/board_history.py`
- Modify: `daily/delta.py`（`load_recent_board_keys` 改委托）
- Test: `tests/test_board_history.py`
- Modify: `.env.example`
**Interfaces:**
- Consumes: `OUTPUT_DIR`、现有 `delta.skill_id` / repo key 约定
- Produces:
- `board_dedup_days() -> int`（默认 7）
- `board_pool_size() -> int`（默认 `max(50, env WECOM_SKILL_POOL)`）
- `featured_dedup_days() -> int`（默认 30）
- `theme_ban_days() -> int`（默认 7）
- `narrative_axis_days() -> int`（默认 3）
- `news_backfill_enabled() -> bool`（默认 False；env `DAILY_NEWS_BACKFILL`）
- `extract_shown_keys(board: str, items: list[dict]) -> list[str]`
- `load_recent_shown_keys(date_str: str, *, lookback_days: int | None = None) -> dict[str, set[str]]`
- `merge_wecom_shown_into_data(data: dict, shown: dict[str, list[str]]) -> dict`
- [ ] **Step 1: Write the failing test**
```python
# tests/test_board_history.py
from __future__ import annotations
import json
import os
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from daily.board_history import extract_shown_keys, load_recent_shown_keys, merge_wecom_shown_into_data
from daily.config import board_dedup_days, news_backfill_enabled
class ConfigDiversityTests(unittest.TestCase):
def test_board_dedup_days_default(self):
with patch.dict(os.environ, {}, clear=True):
self.assertEqual(board_dedup_days(), 7)
def test_news_backfill_default_off(self):
with patch.dict(os.environ, {}, clear=True):
self.assertFalse(news_backfill_enabled())
class ShownKeysTests(unittest.TestCase):
def test_extract_github_repo_keys(self):
items = [{"repo": "a/b"}, {"repo": "c/d"}]
self.assertEqual(extract_shown_keys("github_trending", items), ["a/b", "c/d"])
def test_load_recent_reads_wecom_shown_not_baseline(self):
with tempfile.TemporaryDirectory() as tmp:
out = Path(tmp)
# 前日：shown 只有 x/y；baseline raw 含 a/b —— 周去重只能看到 x/y
payload = {
"data": {
"date": "2026-07-13",
"movement_baseline": {
"github_trending": [{"repo": "a/b"}, {"repo": "x/y"}],
},
"wecom_shown_keys": {"github_trending": ["x/y"]},
}
}
(out / "2026-07-13.data.json").write_text(
json.dumps(payload, ensure_ascii=False), encoding="utf-8"
)
with patch("daily.board_history.OUTPUT_DIR", out):
keys = load_recent_shown_keys("2026-07-14", lookback_days=7)
self.assertEqual(keys["github_trending"], {"x/y"})
self.assertNotIn("a/b", keys["github_trending"])
def test_merge_shown_does_not_touch_baseline(self):
data = {
"movement_baseline": {"github_trending": [{"repo": "raw/one"}]},
}
merged = merge_wecom_shown_into_data(
data, {"github_trending": ["shown/one"]}
)
self.assertEqual(
merged["movement_baseline"]["github_trending"][0]["repo"], "raw/one"
)
self.assertEqual(merged["wecom_shown_keys"]["github_trending"], ["shown/one"])
```
- [ ] **Step 2: Run test to verify it fails**
Run: `pytest tests/test_board_history.py -v`
Expected: FAIL（模块/函数不存在）
- [ ] **Step 3: Implement config + board_history + delta 委托**
在 `daily/config.py` 追加：
```python
def board_dedup_days() -> int:
return max(1, env_int("DAILY_BOARD_DEDUP_DAYS", 7))
def board_pool_size() -> int:
fallback = env_int("DAILY_WECOM_SKILL_POOL", 50)
return max(1, env_int("DAILY_BOARD_POOL_SIZE", max(50, fallback)))
def featured_dedup_days() -> int:
return max(1, env_int("DAILY_FEATURED_DEDUP_DAYS", 30))
def theme_ban_days() -> int:
return max(1, env_int("DAILY_THEME_BAN_DAYS", 7))
def narrative_axis_days() -> int:
return max(1, env_int("DAILY_NARRATIVE_AXIS_DAYS", 3))
def news_backfill_enabled() -> bool:
return env_bool("DAILY_NEWS_BACKFILL", False)
```
新建 `daily/board_history.py`：实现 `BOARD_KEYS` 与 `delta.RECENT_BOARD_KEYS` 同五榜；Skills 用 `delta.skill_id`；GitHub 用 `repo`；`load_recent_shown_keys` **只**读各日 `data.wecom_shown_keys`，缺省空集，读写失败打 log 后当空集。
修改 `daily/delta.py` 的 `load_recent_board_keys`：改为
```python
def load_recent_board_keys(date_str: str, *, lookback_days: int | None = None) -> dict[str, set[str]]:
from daily.board_history import load_recent_shown_keys
from daily.config import board_dedup_days
days = lookback_days if lookback_days is not None else board_dedup_days()
return load_recent_shown_keys(date_str, lookback_days=days)
```
删除（或不再走）原「从 movement_baseline 抽 keys」逻辑，避免 pad 继续把 raw Top 当展示史。
`.env.example` 追加注释块：
```env
# 多样性 / 去重（见 docs/superpowers/specs/2026-07-14-wecom-diversity-dedup-design.md）
# DAILY_BOARD_DEDUP_DAYS=7
# DAILY_BOARD_POOL_SIZE=50
# DAILY_FEATURED_DEDUP_DAYS=30
# DAILY_THEME_BAN_DAYS=7
# DAILY_NARRATIVE_AXIS_DAYS=3
DAILY_NEWS_BACKFILL=0
```
- [ ] **Step 4: Run tests**
Run: `pytest tests/test_board_history.py tests/test_wecom_delta.py -v`
Expected: `test_board_history` PASS；既有 delta 测试若依赖「baseline 即 recent」行为，按 Task 1 语义改断言为 shown_keys（本 Task 内修回归，勿留红）。
- [ ] **Step 5: Commit**
```bash
git add daily/config.py daily/board_history.py daily/delta.py .env.example tests/test_board_history.py tests/test_wecom_delta.py
git commit -m "feat: 拆分 wecom_shown_keys 与 movement_baseline 历史层"
```
---
### Task 2: `board_select` 周去重 + 深池
**Files:**
- Create: `daily/board_select.py`
- Test: `tests/test_board_select.py`
**Interfaces:**
- Consumes: `extract_shown_keys` / `skill_id`；`group_skills_by_source`（Skills 板）
- Produces:
- `board_select(*, board: str, items: list[dict], recent_keys: set[str], limit: int, pool_size: int, kind: Literal["skill","github"]) -> list[dict]`
- 日志短榜：`board_short:{board}:{n}`（`logging.getLogger(__name__).info`）
- [ ] **Step 1: Write the failing test**
```python
# tests/test_board_select.py
from __future__ import annotations
import unittest
from daily.board_select import board_select
def _gh(repo: str) -> dict:
return {"repo": repo, "description": repo}
class BoardSelectTests(unittest.TestCase):
def test_filters_recent_and_keeps_order(self):
pool = [_gh(f"o/r{i}") for i in range(20)]
recent = {"o/r0", "o/r1", "o/r2"}
out = board_select(
board="github_trending",
items=pool,
recent_keys=recent,
limit=5,
pool_size=20,
kind="github",
)
keys = [x["repo"] for x in out]
self.assertEqual(keys, ["o/r3", "o/r4", "o/r5", "o/r6", "o/r7"])
def test_deep_pool_fills_after_filter(self):
pool = [_gh(f"o/r{i}") for i in range(8)]
recent = {f"o/r{i}" for i in range(6)} # 前 6 全封
out = board_select(
board="github_emerging",
items=pool,
recent_keys=recent,
limit=5,
pool_size=8,
kind="github",
)
self.assertEqual([x["repo"] for x in out], ["o/r6", "o/r7"]) # 短榜
def test_skill_uses_skill_id(self):
items = [
{"id": "a/b/s1", "source": "a/b", "title": "s1", "installs": 10},
{"id": "c/d/s2", "source": "c/d", "title": "s2", "installs": 9},
]
out = board_select(
board="skills_trending",
items=items,
recent_keys={"a/b/s1"},
limit=10,
pool_size=50,
kind="skill",
)
self.assertEqual([x["id"] for x in out], ["c/d/s2"])
```
- [ ] **Step 2: Run test to verify it fails**
Run: `pytest tests/test_board_select.py -v`
Expected: FAIL
- [ ] **Step 3: Implement `board_select`**
```python
# daily/board_select.py — 核心逻辑示意
def board_select(*, board, items, recent_keys, limit, pool_size, kind):
if kind == "skill":
from daily.skills_group import group_skills_by_source
pool = group_skills_by_source(items, limit=pool_size, pool_size=pool_size)
def key_fn(x): return skill_id(x)
else:
pool = items[: max(pool_size, limit)]
def key_fn(x): return str(x.get("repo") or "")
out = []
for item in pool:
k = key_fn(item)
if not k or k in recent_keys:
continue
out.append(item)
if len(out) >= limit:
break
if len(out) < limit:
logger.info("board_short:%s:%s", board, len(out))
return out
```
Skills：输入可为未 group 的 raw；函数内 group。GitHub：输入为 repo 列表。
- [ ] **Step 4: Run tests — expect PASS**
Run: `pytest tests/test_board_select.py -v`
- [ ] **Step 5: Commit**
```bash
git add daily/board_select.py tests/test_board_select.py
git commit -m "feat: 实现 board_select 周去重与深池补满"
```
---
### Task 3: 接入 generate / format_wecom（唯一列表主人 + post-render 写回）
**Files:**
- Modify: `daily/generate.py`（选榜、传入 pad、渲染后 merge shown）
- Modify: `daily/format_wecom.py`（delta pad 已通过改写后的 `load_recent_board_keys` 读 shown；确保传入的 `*_pad` 池已是 `board_select` 深池结果）
- Modify: `daily/report_data.py`（可选：llm_input 切片改为 board_select 后列表，避免 Agent 看见未去重 Top）
- Test: `tests/test_board_history.py` 增补「shown ≠ baseline 推导」集成断言；`tests/test_wecom_delta.py` pad 用例
**Interfaces:**
- Consumes: Task1–2
- Produces: 每次成功 generate 后 `output/{date}.data.json` 含 `data.wecom_shown_keys`
- [ ] **Step 1: Write / extend failing integration test**
```python
def test_persist_shown_keys_differs_from_baseline_keys(self):
# 构造：raw trending 前 3 名本周已 shown；board_select 选出 3..；
# movement_baseline 仍含 0..compare_depth
# 断言 data["wecom_shown_keys"]["github_trending"] 与 baseline repos 集合不等
...
```
（可用临时 `OUTPUT_DIR` + 调用抽取出的 `persist` 辅助，或测 `merge_wecom_shown_into_data` + `board_select` 组合。）
- [ ] **Step 2: Run — expect FAIL（generate 尚未写 shown）**
- [ ] **Step 3: Wire generate**
在 `generate_report` 中，在组装 wecom 榜之前：
1. `recent = load_recent_shown_keys(date_str)`
2. 对五榜分别 `board_select(...)` 得到 `selected_*`（limit=wecom_*，pool=`board_pool_size()`）
3. full：渲染用 `selected_*`
4. delta：`trending_pad`/`github_*_pad` = 同规则更大 pool 的 select 结果（或 raw 深池再 select）；`replace_wecom_board_sections(..., pad=True)` 内部 recent 已是 shown
5. 渲染后根据**最终写入正文的 items**（full=selected；delta=函数返回或并行计算最终列表）调用 `extract_shown_keys`，`merge_wecom_shown_into_data`，写回 data.json（在现有 `save_json` 路径合并字段）
注意：`movement_baseline` 仍用 **raw** compare 切片构建（`report_data.build_llm_input` 现逻辑保留）。
若 `build_llm_input` 当前把未过滤 Top 塞进 Agent：改为传入 `selected_*`（或另字段 `boards_for_wecom`），避免 opening 引用已周封杀的榜首。
辅助：在 `format_wecom` 增加 `resolve_wecom_board_items(...)` 返回最终 items dict，供写回与 featured 池 A 共用，避免正文与 history 分叉。
- [ ] **Step 4: Run tests**
Run: `pytest tests/test_board_select.py tests/test_board_history.py tests/test_wecom_delta.py -v`
Expected: PASS
- [ ] **Step 5: Commit**
```bash
git add daily/generate.py daily/format_wecom.py daily/report_data.py tests/
git commit -m "feat: generate 以 board_select 为唯一列表主人并写回 shown keys"
```
---
### Task 4: `featured_resolve`（先定人再 research）
**Files:**
- Modify: `daily/featured_pick.py`
- Modify: `daily/generate.py`（调用顺序）
- Test: `tests/test_featured_resolve.py`
**Interfaces:**
- Consumes: 最终展示 items（池 A）、raw 深池（池 B）、近 30 日 `featured_pick_key`
- Produces:
- `featured_identity_key(featured: dict) -> str`（`type==skill` → id；github → repo；兜底 url path）
- `load_recent_featured_keys(date_str, days) -> set[str]`
- `featured_resolve(*, date_str, candidate: dict | None, pool_a: list[dict], pool_b: list[dict], rng: random.Random | None) -> tuple[dict | None, str | None]`
返回 `(resolved_seed_or_featured_stub, identity_key)`；**不含**完整 why（why 由后续 research 写）
- 改 `apply_featured_pick`：先 resolve 身份（若与昨日冲突则换候选写入 query），再 `research_featured_pick`
- [ ] **Step 1: Failing tests**
```python
# tests/test_featured_resolve.py
def test_same_as_yesterday_picks_from_pool_a(self):
yesterday_key = "headroomlabs-ai/headroom"
pool_a = [
{"repo": "headroomlabs-ai/headroom", "board": "github_topic"},
{"repo": "ollama/ollama", "board": "github_trending"},
]
rng = random.Random(0)
resolved, key = featured_resolve(
date_str="2026-07-14",
candidate={"type": "github", "url": "https://github.com/headroomlabs-ai/headroom", "title": "headroom"},
pool_a=pool_a,
pool_b=[],
recent_featured={yesterday_key},
yesterday_key=yesterday_key,
rng=rng,
)
self.assertNotEqual(key, yesterday_key)
self.assertEqual(key, "ollama/ollama")
def test_pool_a_before_pool_b(self):
...
def test_exhausted_keeps_original(self):
...
```
- [ ] **Step 2: Run — FAIL**
- [ ] **Step 3: Implement**
`featured_resolve`：若无 candidate 或与 `yesterday_key` 不同 → 原样返回。
冲突时：过滤 `recent_featured | {yesterday_key}`，先从 pool_a 建可选项（每项抽 identity），`rng.choice`；空则 pool_b；仍空 log `featured_fallback_exhausted` 并保留原 candidate。
`apply_featured_pick` / generate 流程：
1. 解析 env 得到初始 query/candidate
2. `featured_resolve`（此时池 A 已是 board_select 结果）
3. 若换人：用新 repo/skill 构造 config，再 `research_featured_pick`
4. 写入 `llm_input["featured_pick"]` 与之后 data.`featured_pick_key`
随机默认：`random.Random(int(hashlib.sha256(f"{date_str}:featured".encode()).hexdigest()[:16], 16))`
- [ ] **Step 4: pytest PASS**
- [ ] **Step 5: Commit**
```bash
git add daily/featured_pick.py daily/generate.py tests/test_featured_resolve.py
git commit -m "feat: 首推与昨日冲突时改推并保证一月不重复"
```
---
### Task 5: `narrative_axis` 硬互斥 + Step1 软禁 theme
**Files:**
- Create: `daily/narrative_axis.py`
- Modify: `daily/agent_workflow.py`（`analyze_trends`）
- Modify: `skills/daily-agent/SKILL.md`
- Modify: `daily/generate.py`（落盘 `narrative_axis`；注入 llm_input）
- Test: `tests/test_narrative_axis.py`
**Interfaces:**
- Produces:
- `NARRATIVE_AXES: tuple[str, ...] = ("政策监管", "模型发布", "工具链/Agent", "芯片算力", "开源生态", "应用落地", "安全/诉讼")`
- `pick_narrative_axis(used: set[str], *, rng: random.Random | None = None) -> str`
- `load_recent_axes(date_str, days) -> list[str]`（近 N 日 data.`narrative_axis`）
- `enforce_narrative_axis(trends: dict, axis: str) -> dict`（强制 trends["narrative_axis"]=axis）
- [ ] **Step 1: Failing tests**
```python
def test_pick_excludes_used(self):
used = {"政策监管", "模型发布", "工具链/Agent"}
for _ in range(20):
axis = pick_narrative_axis(used, rng=random.Random(1))
self.assertNotIn(axis, used)
def test_enforce_overwrites_llm(self):
trends = {"narrative_axis": "开源生态", "opening": "..."}
out = enforce_narrative_axis(trends, "芯片算力")
self.assertEqual(out["narrative_axis"], "芯片算力")
```
- [ ] **Step 2: FAIL → Step 3 implement**
`analyze_trends`：计算 `axis = pick_narrative_axis(set(load_recent_axes(...)))`；把 `required_narrative_axis` 与近 `theme_ban_days` 的 theme/opening 摘要列表注入 system prompt；要求 JSON 含 `narrative_axis` 且必须等于 required。解析后 `enforce_narrative_axis`。
generate 将 axis 写入 data.json。
SKILL.md Step1 schema 增加 `narrative_axis` 字段说明。
- [ ] **Step 4–5: pytest + commit**
```bash
git commit -m "feat: 代码选定叙事轴并注入 Agent 开场约束"
```
---
### Task 6: 取消新闻「放宽窗口」
**Files:**
- Modify: `daily/news/fetch.py`（`_apply_pushed_dedup_with_backfill`）
- Modify: `daily/news/research.py`（确认只 filter_unpushed；不足不拉超窗）
- Create: `daily/news/sanitize.py`（或放入 `text_utils`）— `strip_relax_window_prefix(text: str) -> str`
- Modify: `skills/daily-ai-news-research/SKILL.md`（删除「放宽至 48h 并注明」；改为不足则少返回、禁止标注）
- Modify: `daily/generate.py` / news finalize（对 desc_short 剥前缀）
- Test: `tests/test_news_relax.py`；扩展 `tests/test_news_fetch_window.py`
**Interfaces:**
- `news_backfill_enabled() == False` 时：`_apply_pushed_dedup_with_backfill` 等价于只返回 `filter_unpushed_items(...)[:limit]`，**不**再从 `picked` 塞回
- `strip_relax_window_prefix`：去掉开头的 `放宽窗口[:：]?` / `放宽至[^：:]*[:：]`
- [ ] **Step 1: Failing tests**
```python
def test_backfill_disabled_does_not_reinsert_pushed(self):
# fresh 不足 limit；picked 含已推；BACKFILL=0 → 结果不含已推 link
...
def test_strip_relax_prefix(self):
self.assertEqual(
strip_relax_window_prefix("放宽窗口：苹果起诉 OpenAI"),
"苹果起诉 OpenAI",
)
```
- [ ] **Step 2–4: 实现并跑 `pytest tests/test_news_relax.py tests/test_news_fetch_window.py -v`**
Research 路径：SKUILL 改完后，代码侧对 items 统一 `strip`；不足时 log `news_short:{n}`，接受短列表。
- [ ] **Step 5: Commit**
```bash
git commit -m "fix: 关闭新闻放宽凑数并剥离放宽窗口文案"
```
---
### Task 7: 端到端回归与文档对齐
**Files:**
- Modify: 如有遗漏的 `.env.example` / SKILL
- Test: 全量相关测试
- [ ] **Step 1: 跑全套**
Run:
```bash
pytest tests/test_board_history.py tests/test_board_select.py tests/test_featured_resolve.py tests/test_narrative_axis.py tests/test_news_relax.py tests/test_news_fetch_window.py tests/test_wecom_delta.py tests/test_featured_pick.py -v
```
Expected: 全部 PASS
- [ ] **Step 2: Spec 对照清单（人工）**
| Spec 要求 | 任务 |
|-----------|------|
| wecom_shown_keys ≠ movement_baseline | T1, T3 |
| board_select 唯一主人 + delta pad 共用 shown | T2, T3 |
| 首推月去重 A→B、先定人再 why | T4 |
| narrative_axis 硬保证 | T5 |
| 禁放宽 backfill + 剥前缀 + hours 窗 | T6 |
| 成功标准可测 | 各测覆盖 |
- [ ] **Step 3: Commit（若有收尾文档）**
```bash
git add -u
git commit -m "test: 多样性去重全链路回归通过"
```
---
## Spec Coverage Self-Review
| Spec 节 | 计划任务 |
|---------|----------|
| 1.1 双基准分离 | T1 |
| 1.2 唯一列表主人 | T2–T3 |
| 1.3 真相表 | T1, T3–T5 落盘字段 |
| 2.1 board_select | T2 |
| 2.2 featured_resolve | T4 |
| 3.1 narrative_axis + 软 theme | T5 |
| 3.2 取消放宽 | T6 |
| 4.x 配置/降级/测试 | T1–T7 |
无 TBD；Commit 信息均为中文描述体。
## Execution Handoff
Plan complete and saved to `docs/superpowers/plans/2026-07-14-wecom-diversity-dedup.md`.
**两种执行方式：**
1. **Subagent-Driven（推荐）** — 每任务新开子代理，任务间审查
2. **Inline Execution** — 本会话按 `executing-plans` 连续做完，设检查点
要哪个？

271
docs/superpowers/specs/2026-07-14-wecom-diversity-dedup-design.md Normal file
View File

@@ -0,0 +1,271 @@
# Design: 企微早报多样性与去重
Generated: 2026-07-14
Repo: daily-robots
Status: APPROVED
Mode: Builder
Related: `docs/design-wecom-delta-mode.md`（Delta 列表模式）
Revision: office-hours A —— 拆分展示历史、单一列表主人、历史真相表（2026-07-14）
## Problem Statement
近两日企微早报（如 2026-07-13 / 07-14）骨架相同，且：
- **今日首推**连续两天同为 `headroom`
- **开场主题**同属「上下文压缩 + 视频/Skills」腔调
- **AI 时讯**出现「放宽窗口」旧闻凑数标注
- **Skills / GitHub 各榜**（尤其新兴榜）周内大量重复展示
读者需要「今天新信息」，而不是换日期的复印机。
## Decisions（已确认）
| 决策点 | 选择 |
|--------|------|
| 实现路径 | **A：管线选择器**（代码硬保证去重；LLM 只写开场/理由/摘要；中文化仍走现有 `localize`） |
| Skills「同一类」 | 暂不管；沿用现有 `group_skills_by_source`（**已知残留**：同 source 换 skill id 仍可能周内再出现） |
| 周去重后不足 Top N | **深池补满**，仍保证周内未出现；池空则短榜，不破周约束 |
| 首推改推候选 | **先展示榜（池 A），再 raw 深池（池 B）**；一月内首推不重复 |
| 开场主题 | **近 7 天禁主题/句式（软）+ 叙事轴与近 3 天不同（硬，代码选轴）** |
| 取消「放宽窗口」 | **禁止旧闻/已推凑数**；不够则深度检索补新闻；禁止任何「放宽」文案标注；仍不够则短列表 |
| 展示历史 vs 异动基准 | **必须拆开**：`movement_baseline` ≠ `wecom_shown_keys` |
| full/delta 列表主人 | **唯一主人** = `board_select`（含 full 与 delta 的 moves∪pad）；禁止二次独立选榜 |
## Explicit Non-Goals
| 项 | 状态 |
|----|------|
| 语义级 Skill「同一类」分类 | ❌ 本期不做 |
| 同日 Skills Trending ↔ Hot 互斥 | ❌ 本期不做 |
| 独立 editorial 微服务 | ❌ 不做 |
| 改写 `localize` 为脚本机翻 | ❌ 保持 LLM + 缓存 |
| 编辑指定首推豁免改推（`FEATURED_FORCE`） | ❌ 本期不做 |
| 「开场关键短语」硬匹配算法 | ❌ 本期不做（仅 prompt 软约束；不进硬成功标准） |
## Recommended Approach: 管线选择器（路径 A）
在现有 `daily generate` 内增加选条/裁决层，不新起进程：
```
采集 raw 榜 + 新闻
→ build movement_baseline（raw Top compare_depth —— 仅供次日「新入榜」，禁止写展示历史）
→ board_select（读 wecom_shown_keys 周历史；周去重 + 深池；输出当日最终展示列表）
· full：直接取 board_select 结果前 N
· delta：在 board_select 候选池内做 moves∪pad（pad 也只从该池/同规则深池取，不再另起一套历史）
→ featured_resolve（与昨日首推相同则改推；池 A = 本 run 最终展示 keys）
→ research why（先定人，再写 why_today）
→ news_select（禁放宽凑数；深检索补满；剥「放宽」前缀）
→ editorial（代码选 narrative_axis；prompt 附近 7 天 theme 软禁）
→ 渲染 wecom
→ 写回 wecom_shown_keys = 最终进入企微正文的榜条目 keys（post-render）
→ 其余 history（首推月、axis）写入 data.json 约定字段
```
**LLM 负责**：`opening` / `theme_line`、首推 `why_today`、新闻与榜单项中文摘要（`localize`）。
**代码负责**：谁上榜、首推换谁、周/月去重、`narrative_axis` 选取、是否允许旧闻。
### Approaches Considered
| | A 管线选择器（采用） | B 偏 LLM 约束 | C 独立 editorial 服务 |
|--|--|--|--|
| 优点 | 可测；与 pushed-links 模式一致 | 改 prompt 快 | 边界清晰 |
| 缺点 | 需动 generate / featured / news / format | 易漏、难测 | 过重 |
---
## Section 1 — 总览、列表主人、历史真相表
### 1.1 两种「基准」禁止混用
| 字段 | 含义 | 写入时机 | 读者 |
|------|------|----------|------|
| `movement_baseline` | **Raw** 各榜 Top `compare_depth`（现网语义不变） | `build_llm_input` / 采集后尽早 | `build_movement_context`（新入榜） |
| `wecom_shown_keys` | **读者实际见到**的各榜 key 集合（及可选 rank） | **wecom 渲染完成之后** | `board_select` 周去重；delta pad；测试 |
**禁止**：把 `wecom_shown_keys` 写入或覆写 `movement_baseline`。
**禁止**：让 `load_recent_board_keys` 继续读 `movement_baseline` 充当「已展示」——应改为读近 N 日 `wecom_shown_keys`（可保留函数名，换数据源；或新建 `load_recent_shown_keys`）。
### 1.2 唯一列表主人
`board_select`（模块可挂在 `daily/board_select.py` 或扩 `delta.py`）是各榜**最终展示行**的唯一生产者：
| 模式 | 行为 |
|------|------|
| `full` | `board_select(raw, shown_history) →` 至多 N 条，直接渲染 |
| `delta` | 先算相对 `movement_baseline` 的 moves；展示 = `moves`（已在候选内）∪ `pad`；**pad 候选必须来自同一周去重池**（与 full 同一套 `board_select` 规则），不得再读 raw baseline 当「已展示」 |
交互影响（非「完全正交」）：周去重会减少可展示重复项 → delta 日可能更短、silent/gate 行为可能变化。`DAILY_WECOM_MODE` 枚举语义不变，但列表密度会变。
### 1.3 历史真相表（单一来源）
全部落在 `output/{date}.data.json`（新闻 pushed-links 例外，沿用现网 cache）。
| 字段路径 | 窗口 | Key 规则 | 写者 | 读者 |
|----------|------|----------|------|------|
| `data.movement_baseline` | 次日对比用 | raw 条目切片 | `build_movement_baseline` | movement |
| `data.wecom_shown_keys.{board}` | 滚动 7 天（读近 7 日文件） | Skills：与现网 `_skill_keys_in_board_item` / `skill_id` 一致；GitHub：`owner/repo` | post-render persist | `board_select` / pad |
| `data.featured_pick_key` | 滚动 30 天 | skill id 或 `owner/repo` | `featured_resolve` 成功后 | 月去重 |
| `data.narrative_axis` | 滚动 3 天 | 枚举字符串 | 代码 `pick_narrative_axis` | Step 1 约束 / 校验 |
| `data.theme_line` / trends opening | 近 7 日供 prompt | 原文 | editorial 落盘 | Step 1 软禁（不硬匹配） |
| `CACHE_DIR/pushed-news-links.json` | `DAILY_NEWS_DEDUP_DAYS` | 规范化 URL | 推送成功后 | news filter |
不另建平行 CACHE「board-history.json」，避免双源漂移。冷启动：缺文件 = 空集合。
### 1.4 数据流挂点
| 逻辑 | 挂点 |
|------|------|
| `movement_baseline` | 现网：raw 榜入库时（不变） |
| `board_select` | 渲染前；输出写入供 Agent/`llm_input` 与 wecom 共用的最终列表字段 |
| delta pad | **调用同一周去重历史**（`wecom_shown_keys`），不再独立解释 `movement_baseline` 为展示史 |
| `featured_resolve` | **先于** why 检索；池 A = 本 run `board_select`（delta 则为本 run 最终展示列表） |
| 新闻 | 所有 prepare 路径关 backfill；research SKILL 改文案规则；后处理剥「放宽」 |
| `narrative_axis` | 代码先选轴再注入 Step 1；LLM 不得另选冲突轴 |
| `wecom_shown_keys` 写回 | `replace_wecom_*` / `build_wecom_report` 之后，与最终正文列表一致 |
---
## Section 2 — 各榜选条 + 今日首推改推
### 2.1 `board_select`（五榜共用）
适用：`skills_trending` / `skills_hot` / `github_trending` / `github_emerging` / `github_topic`。
```
输入：当日 raw 池（深，pool ≥ DAILY_BOARD_POOL_SIZE）
历史：近 DAILY_BOARD_DEDUP_DAYS 的 wecom_shown_keys[board]
输出：至多 N 条（N = 现有 wecom Top 配置）
1. 现有整理（Skills：source 合并；GitHub：repo key）
2. 滤掉近 7 天该榜 wecom_shown_keys
3. 按原排名取前 N
4. 不足 → 继续扫深池，仍排除周历史，直到满 N 或池空
5. 池空仍不足 → 短榜；日志 board_short:{board}:{n}；不回填周内已展示条目
```
分榜独立历史：Trending 出过的 skill，Hot 仍可出。
Post-render：将**实际写入企微的** keys 写入当日 `wecom_shown_keys`（测试断言：history ⊆ / == 渲染列表，**≠** `movement_baseline`）。
### 2.2 `featured_resolve`
**触发**：本 run 拟用首推身份与**前一天** `featured_pick_key`（或等价 data 字段）相同。
身份函数：skill → `skill_id`；github → `owner/repo`。
含 `DAILY_FEATURED_PICK` 与自动首推；本期不豁免。无昨日文件 → 不改推。
**顺序（硬）**：定候选 → 再 `research`/`why_today`（禁止先写旧条目 why 再改人却不重写）。
**候选**：
1. **池 A**：本 run **最终会展示**的 Skills + GitHub 榜条目（与 `wecom_shown_keys` 同源结构）
2. **池 B**：raw 深池中尚未进入本 run 展示者
**过滤**：近 30 天 `featured_pick_key`；排除冲突项自身。
**抽取**：`hash(date_str + "featured")` 可复现；测试可注入 RNG。先 A 后 B；仍空 → 保留原首推 + `featured_fallback_exhausted`。
**落盘**：`data.featured_pick_key`。
---
## Section 3 — 开场主题 + AI 时讯
### 3.1 开场主题
| 机制 | 强度 | 规则 |
|------|------|------|
| `narrative_axis` | **硬** | 代码 `pick_narrative_axis(used_last_N)` 从剩余枚举选取；注入 prompt；LLM 输出须等于该轴；冲突则重试 1 次，再失败则**强制覆写为代码所选轴**再落盘（保证成功标准可测） |
| theme/opening 软禁 | **软** | prompt 附近 7 天 `theme_line`/opening 摘要；禁止复述；**无** n-gram 硬匹配；**不**列入硬成功标准 |
**叙事轴枚举**：
`政策监管` · `模型发布` · `工具链/Agent` · `芯片算力` · `开源生态` · `应用落地` · `安全/诉讼`
`opening` 首句证据须来自当日数据；首推改推后须跟新首推或当日主轴新闻。
### 3.2 AI 时讯：取消「放宽窗口」
目标条数 = 现网配置之和（如 `DAILY_WECOM_AI_NEWS` + tech/CN 等），文档不写死「15」。
1. **所有 prepare 路径**关闭「不够塞回已推/旧条」（`DAILY_NEWS_BACKFILL=0` 默认）；`pushed-news-links` 过滤保留。
2. 不够 → 深度检索补新闻（须：https link、未 pushed、可核实）；**补入年龄上限** = `DAILY_AI_NEWS_HOURS`（与主窗一致），禁止借 research 变相放宽到任意旧闻。
3. 改 research SKILL：删除「放宽至 48h 并注明」；后处理剥 `放宽窗口`/`放宽至` 前缀或丢弃。
4. 仍不足 → 短列表 + `news_short:{n}`。
中文化：`daily/localize.py`（不变）。
---
## Section 4 — 配置、错误处理、测试
### 4.1 环境变量
| 变量 | 默认 | 含义 |
|------|------|------|
| `DAILY_BOARD_DEDUP_DAYS` | `7` | 读 `wecom_shown_keys` 的滚动天数 |
| `DAILY_BOARD_POOL_SIZE` | ≥50 / 与现有 skill pool 对齐 | 深池扫描深度 |
| `DAILY_FEATURED_DEDUP_DAYS` | `30` | 今日首推月去重 |
| `DAILY_THEME_BAN_DAYS` | `7` | 软禁：注入 prompt 的 theme 天数 |
| `DAILY_NARRATIVE_AXIS_DAYS` | `3` | 叙事轴互斥窗 |
| `DAILY_NEWS_BACKFILL` | `0` | `0`=禁止旧闻凑数 |
| `DAILY_NEWS_DEDUP_DAYS` | 已有 `7` | pushed-links |
`DAILY_DELTA_PAD_LOOKBACK_DAYS` 应与 `DAILY_BOARD_DEDUP_DAYS` 对齐，且 **pad 与 board_select 共用 `wecom_shown_keys`**（窗口对齐不够，数据源必须同一）。
### 4.2 错误与降级
| 情况 | 行为 |
|------|------|
| 无 `wecom_shown_keys` 历史 | 空集合，正常满榜 |
| 周去重后深池不足 | 短榜 + `board_short` |
| 首推冲突且 A/B 空 | 保留原首推 + `featured_fallback_exhausted` |
| LLM 轴与代码轴冲突 | 覆写为代码轴 |
| 深检索仍不足时讯 | 短列表；禁止 backfill |
| history 读写失败 | 当次按空历史 + error 日志 |
### 4.3 测试（pytest）
1. `board_select`：假 `wecom_shown_keys` + 深池 → 无周交集；深池补满；不足短榜
2. **回归钉死**：写回后 `wecom_shown_keys` ≠ 用 `movement_baseline` 推导的集合（构造 raw Top 与展示 Top 故意不同）
3. delta：pad 不引入近 7 日 `wecom_shown_keys` 内 key
4. `featured_resolve`：先定人再 why；A 优先 B；月未见；可注入 RNG
5. news：`BACKFILL=0`；剥「放宽*」；research 补入不超 hours 窗
6. `pick_narrative_axis`：近 3 天互斥；落盘轴 == 代码轴
7. 既有 delta / pushed_links / wecom 回归不挂
### 4.4 成功标准（硬）
- 连续两天：**首推 key 不同**（除非 `featured_fallback_exhausted`）
- 同一榜近 7 日 `wecom_shown_keys`：**无重复 key**（池足够时）
- 时讯：无「放宽*」标注；无 backfill 已推 link
- 近 3 天 `narrative_axis`：**两两不同**（代码保证）
软标准（不闸门）：opening 读感不像连续复印。
---
## Implementation Sketch（非计划明细）
1. `data.json` 增加 `wecom_shown_keys`；改 `load_recent_*` 数据源
2. `board_select` + 让 delta pad 共用
3. post-render persist shown keys
4. `featured_resolve` 时序修正
5. news backfill off + SKILL + 剥前缀
6. `pick_narrative_axis` + prompt 注入
7. 测试如上
正式任务拆解 → `writing-plans`。
## Office-hours Review Notes
- 对抗审阅质量约 4/10 → 本修订处理三大硬伤（存储拆分、列表主人、真相表）。
- 未纳入本期（原选项 B）：首推质量加权、关键短语硬匹配。
- 已知残留：source 级「同类」周内可再现。
## Spec Self-Review
- [x] `movement_baseline` 与 `wecom_shown_keys` 职责分离写死
- [x] 单一列表主人 + full/delta 交互说明
- [x] 历史真相表无「与/或」双源
- [x] 轴硬 / 短语软；成功标准不含无法验证的短语匹配
- [x] 周不足=深池、首推=A→B、新闻禁放宽 与访谈一致

43
run-scheduler.ps1 Normal file
View File

@@ -0,0 +1,43 @@
# Start the daily report scheduler (generate @ 08:50, push @ 09:00 by default)
# Usage:
# .\run-scheduler.ps1
# .\run-scheduler.ps1 -DryRun
# .\run-scheduler.ps1 -Once
param(
[switch]$DryRun,
[switch]$Once
)
$ErrorActionPreference = "Stop"
$Root = Split-Path -Parent $MyInvocation.MyCommand.Path
Set-Location $Root
function Import-DotEnvFile {
param([string]$Path)
if (-not (Test-Path $Path)) { return }
Get-Content $Path -Encoding UTF8 | ForEach-Object {
if ($_ -match '^\s*#' -or $_ -notmatch '=') { return }
$pair = $_ -split '=', 2
if ($pair.Count -eq 2) {
$name = $pair[0].Trim()
$value = $pair[1].Trim().Trim('"').Trim("'")
if ($name -and $value) {
Set-Item -Path "Env:$name" -Value $value
}
}
}
}
Import-DotEnvFile (Join-Path $Root ".env")
Import-DotEnvFile (Join-Path $Root ".env.local")
$args = @("python", "-m", "daily", "schedule")
if ($DryRun) { $args += "--dry-run" }
if ($Once) { $args += "--once" }
Write-Host "Starting scheduler: $($args -join ' ')"
& $args[0] $args[1..($args.Length - 1)]
if ($LASTEXITCODE -ne 0) {
throw "scheduler failed with exit code $LASTEXITCODE"
}

87
skills/daily-featured-pick/SKILL.md Normal file
View File

@@ -0,0 +1,87 @@
# 早报今日首推检索
你是 **Skills / GitHub / AI 时讯早报** 的编辑研究员。Python 已完成榜单抓取；你负责为 **编辑指定的今日首推** 收集可核实信息，供后续趋势分析与写稿使用。
## 场景
- 触发：环境变量 `DAILY_FEATURED_PICK` 有值（如 `gstack` 或 `gstack|https://github.com/you/gstack`）
- 输出：严格 JSON，写入 `output/YYYY-MM-DD.featured.json`
- 你 **不** 写整篇早报、 **不** 改榜单顺序、 **不** 推送
## 输入
```json
{
"query": "gstack",
"url_hint": "https://github.com/you/gstack",
"cwd": "D:\\path\\to\\workspace",
"data_matches": {
"skills": [],
"github": []
}
}
```
| 字段 | 含义 |
|------|------|
| `query` | 编辑指定的关键词（skill 名 / repo 名片段） |
| `url_hint` | 可选，项目主页或仓库 URL |
| `cwd` | Cursor 工作目录，可在此检索 README / SKILL.md |
| `data_matches` | Python 已在今日 Top 榜中预匹配的条目（**优先使用其数字**） |
## 检索顺序
1. **读 `data_matches`**：若 skills/github 有匹配， installs / star / repo / link **必须来自此处**，不得改写
2. **读 `cwd` 本地仓库**：搜索 README、SKILL.md、package.json 描述，提炼「做什么 + 技术栈」
3. **用 `url_hint`**：作为项目主页；无本地文件时可仅基于 URL 与 query 写 summary（须标注 evidence 来源）
4. **禁止编造**：未在 data_matches / 本地文件 / url_hint 出现的数字、功能、版本一律不写
## 输出
**只输出一个 JSON 对象**，不要 markdown 围栏，不要解释。
```json
{
"title": "gstack",
"type": "skill|github|other",
"command": "npx skills add owner/repo/skill",
"url": "https://...",
"summary": "2–3 句中文：做什么 + 技术栈/场景",
"why_today": "中文，为什么今天主推（事实 + 一句判断）",
"evidence": ["Skills Trending 匹配 · remotion-render · 22.3K", "README: Agent 工作流 CLI"],
"tags": ["agent", "workflow"]
}
```
### 字段要求
| 字段 | 要求 |
|------|------|
| `title` | 展示名，通常与 query 或匹配条目 title/repo 短名一致 |
| `type` | `skill` = Skills 条目；`github` = 仓库；`other` = 仅关键词/URL |
| `command` | Skill：`npx skills add {source}/{title}`；GitHub：仓库 URL；other：url_hint 或 query |
| `url` | 可点击链接，来自 data_matches.link / repo url / url_hint |
| `summary` | 36–80 字中文，动词开头，说清用途 |
| `why_today` | 40–80 字，「事实/数字 + 判断」，不用空泛形容词；**不得**出现「编辑指定」「编辑首推」等内部流程用语 |
| `evidence` | 2–4 条短字符串，标明信息来源 |
| `tags` | 0–4 个英文或中文关键词 |
### type 与 command 示例
- Skill 匹配：`type=skill`，`command=npx skills add vercel-labs/skills/find-skills`
- GitHub 匹配：`type=github`，`command=https://github.com/openclaw/openclaw`
- 仅关键词：`type=other`，`command` 用 url_hint
## 写作原则
1. **事实优先**：why_today 每条 claim 能在 evidence 或 data_matches 中找到
2. **数字必真**：installs、star 与 data_matches 完全一致
3. **中文叙述**：summary / why_today 全中文；skill/repo 名保留英文
4. **克制**：不写「值得关注」「game-changer」等空话
## 输出前自检
- [ ] 仅有 JSON，无围栏、无前后说明
- [ ] data_matches 有数字时，summary/why_today 已引用
- [ ] command / url 与 type 一致
- [ ] 未编造未检索到的事实

117
tests/test_ai_news_research.py Normal file
View File

@@ -0,0 +1,117 @@
"""AI 时讯 deep-research 解析与企微格式。"""
from __future__ import annotations
import unittest
from daily.format_wecom import _ai_news_lines, replace_wecom_news_sections
from daily.news.research import parse_research_response
class TestAiNewsResearchParse(unittest.TestCase):
def test_parse_items(self):
raw = """
{
"items": [
{
"title": "Apple sues OpenAI",
"link": "https://techcrunch.com/2026/07/10/apple/",
"source_name": "TechCrunch",
"desc_short": "苹果起诉 OpenAI 涉嫌窃取商业机密",
"published_fmt": "07-11 05:00"
}
]
}
"""
items, tech = parse_research_response(raw, limit=10)
self.assertEqual(len(items), 1)
self.assertEqual(items[0]["source_name"], "TechCrunch")
self.assertIn("苹果", items[0]["desc_short"])
self.assertEqual(tech, [])
def test_dedupe_links(self):
raw = """{"items": [
{"title": "A", "link": "https://example.com/a", "source_name": "Ex", "desc_short": "一"},
{"title": "B", "link": "https://example.com/a", "source_name": "Ex", "desc_short": "二"}
]}"""
items, _ = parse_research_response(raw, limit=10)
self.assertEqual(len(items), 1)
def test_parse_tech_items(self):
raw = """{"items": [
{"title": "A", "link": "https://example.com/a", "source_name": "Ex", "desc_short": "一"}
], "tech_items": [
{"title": "B", "link": "https://example.com/b", "source_name": "Ex", "desc_short": "二"}
]}"""
items, tech = parse_research_response(raw, limit=10, tech_limit=5)
self.assertEqual(len(items), 1)
self.assertEqual(len(tech), 1)
self.assertEqual(tech[0]["title"], "B")
class TestMergedWecomNews(unittest.TestCase):
def test_merged_block_format(self):
items = [
{
"title": "Apple sues OpenAI",
"link": "https://techcrunch.com/x",
"source_name": "TechCrunch",
"desc_short": "苹果起诉 OpenAI",
"published_fmt": "07-11 05:00",
}
]
lines = _ai_news_lines(items, merged=True)
self.assertIn("TechCrunch - Apple sues OpenAI", lines[0])
self.assertIn("— 苹果起诉 OpenAI", lines[0])
self.assertNotIn("07-11", lines[0])
def test_merged_with_tech_block(self):
md = """📰 **早报**
📈 **Skills Trending Top 1**
1. skill
"""
main = [
{"title": "Main", "link": "https://example.com/m", "source_name": "Src", "desc_short": "主条", "published_fmt": "07-11"}
]
tech = [
{"title": "Tech", "link": "https://example.com/t", "source_name": "Src2", "desc_short": "技术条", "published_fmt": "07-12"}
]
out = replace_wecom_news_sections(md, ai_news=main, tech_ai_news=tech, merged=True)
self.assertIn("📰 **AI 时讯精选 Top 2**", out)
self.assertNotIn("技术类时讯", out)
self.assertNotIn("🔧", out)
self.assertNotIn("07-11", out)
self.assertNotIn("07-12", out)
self.assertIn("2. [Src2 - Tech]", out)
def test_replace_merged_removes_split_blocks(self):
md = """📰 **早报**
🌍 **国际 AI 时讯 Top 1**
1. [old](https://example.com/old) · `X`
🇨🇳 **国内 AI 时讯 Top 1**
1. [old2](https://example.com/old2) · `Y`
📈 **Skills Trending Top 1**
1. skill
"""
items = [
{
"title": "New story",
"link": "https://example.com/new",
"source_name": "Fortune",
"desc_short": "新故事",
"published_fmt": "",
}
]
out = replace_wecom_news_sections(md, ai_news=items, merged=True)
self.assertIn("📰 **AI 时讯精选 Top 1**", out)
self.assertNotIn("国际 AI 时讯", out)
self.assertNotIn("国内 AI 时讯", out)
self.assertIn("📈 **Skills Trending Top 1**", out)
if __name__ == "__main__":
unittest.main()

48
tests/test_board_history.py
View File

@@ -27,6 +27,19 @@ class ShownKeysTests(unittest.TestCase):
items = [{"repo": "a/b"}, {"repo": "c/d"}] items = [{"repo": "a/b"}, {"repo": "c/d"}]
self.assertEqual(extract_shown_keys("github_trending", items), ["a/b", "c/d"]) self.assertEqual(extract_shown_keys("github_trending", items), ["a/b", "c/d"])
def test_extract_skill_keys_include_source(self):
items = [
{
"id": "open.feishu.cn/lark-drive",
"source": "open.feishu.cn",
"title": "lark-drive",
}
]
self.assertEqual(
extract_shown_keys("skills_trending", items),
["open.feishu.cn/lark-drive", "open.feishu.cn"],
)
def test_load_recent_reads_wecom_shown_not_baseline(self): def test_load_recent_reads_wecom_shown_not_baseline(self):
with tempfile.TemporaryDirectory() as tmp: with tempfile.TemporaryDirectory() as tmp:
out = Path(tmp) out = Path(tmp)
@@ -48,6 +61,41 @@ class ShownKeysTests(unittest.TestCase):
self.assertEqual(keys["github_trending"], {"x/y"}) self.assertEqual(keys["github_trending"], {"x/y"})
self.assertNotIn("a/b", keys["github_trending"]) self.assertNotIn("a/b", keys["github_trending"])
def test_load_recent_falls_back_to_wecom_md_when_shown_missing(self):
"""旧日 data 无 wecom_shown_keys 时，从同日 wecom.md 解析实际展示 keys。"""
with tempfile.TemporaryDirectory() as tmp:
out = Path(tmp)
payload = {
"data": {
"date": "2026-07-13",
"github_trending": [{"repo": "other/top"}],
}
}
(out / "2026-07-13.data.json").write_text(
json.dumps(payload, ensure_ascii=False), encoding="utf-8"
)
(out / "2026-07-13.wecom.md").write_text(
"\n".join(
[
"🐙 **GitHub Trending Top 2**",
"1. [vinta/awesome-python](https://github.com/vinta/awesome-python)",
"2. [react/react](https://github.com/react/react)",
"",
"🌱 **GitHub 新兴 Top 1**",
"1. [elder-plinius/T3MP3ST](https://github.com/elder-plinius/T3MP3ST)",
]
),
encoding="utf-8",
)
with patch("daily.board_history.OUTPUT_DIR", out):
keys = load_recent_shown_keys("2026-07-14", lookback_days=7)
self.assertEqual(
keys["github_trending"],
{"vinta/awesome-python", "react/react"},
)
self.assertEqual(keys["github_emerging"], {"elder-plinius/T3MP3ST"})
self.assertNotIn("other/top", keys["github_trending"])
def test_merge_shown_does_not_touch_baseline(self): def test_merge_shown_does_not_touch_baseline(self):
data = { data = {
"movement_baseline": {"github_trending": [{"repo": "raw/one"}]}, "movement_baseline": {"github_trending": [{"repo": "raw/one"}]},

33
tests/test_board_select.py
View File

@@ -53,6 +53,39 @@ class BoardSelectTests(unittest.TestCase):
) )
self.assertEqual([x["id"] for x in out], ["c/d/s2"]) self.assertEqual([x["id"] for x in out], ["c/d/s2"])
def test_skill_filters_recent_by_source(self):
items = [
{"id": "a/b/s-new", "source": "a/b", "title": "s-new", "installs": 10},
{"id": "c/d/s2", "source": "c/d", "title": "s2", "installs": 9},
]
out = board_select(
board="skills_hot",
items=items,
recent_keys={"a/b"}, # source-level history
limit=10,
pool_size=50,
kind="skill",
)
self.assertEqual([x["id"] for x in out], ["c/d/s2"])
def test_skill_filters_recent_skill_id_as_same_source(self):
from daily.format_wecom import expand_skill_recent_keys
items = [
{"id": "open.feishu.cn/lark-drive", "source": "open.feishu.cn", "title": "lark-drive", "installs": 10},
{"id": "fresh/src/s", "source": "fresh/src", "title": "s", "installs": 9},
]
recent = expand_skill_recent_keys({"open.feishu.cn/lark-doc"})
out = board_select(
board="skills_trending",
items=items,
recent_keys=recent,
limit=10,
pool_size=50,
kind="skill",
)
self.assertEqual([x["id"] for x in out], ["fresh/src/s"])
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()

95
tests/test_featured_pick.py Normal file
View File

@@ -0,0 +1,95 @@
"""Tests for daily.featured_pick."""
from __future__ import annotations
import os
import unittest
from unittest.mock import patch
from daily.featured_pick import (
apply_featured_pick,
match_in_data,
parse_featured_pick,
pick_command_from_featured,
pick_why_from_featured,
research_featured_pick,
)
SAMPLE_INPUT = {
"skills_trending": [
{
"id": "foo/bar/gstack-cli",
"title": "gstack-cli",
"source": "foo/bar",
"installs": 1200,
"installs_fmt": "1.2K",
"link": "https://skills.sh/foo/bar/gstack-cli",
"description": "Agent workflow CLI",
}
],
"skills_hot": [],
"github_trending": [
{
"repo": "acme/gstack",
"url": "https://github.com/acme/gstack",
"total_stars_fmt": "3.2K",
"description": "GStack toolkit",
}
],
"github_emerging": [],
"github_topic": {"topic": "llm", "repos": []},
}
class ParseFeaturedPickTests(unittest.TestCase):
def test_empty(self):
with patch.dict(os.environ, {}, clear=True):
self.assertIsNone(parse_featured_pick())
def test_query_only(self):
with patch.dict(os.environ, {"DAILY_FEATURED_PICK": "gstack"}, clear=True):
self.assertEqual(parse_featured_pick(), {"query": "gstack"})
def test_query_with_url(self):
with patch.dict(os.environ, {"DAILY_FEATURED_PICK": "gstack|https://example.com"}, clear=True):
self.assertEqual(
parse_featured_pick(),
{"query": "gstack", "url_hint": "https://example.com"},
)
class MatchInDataTests(unittest.TestCase):
def test_matches_skill_and_github(self):
matches = match_in_data(SAMPLE_INPUT, "gstack")
self.assertEqual(len(matches["skills"]), 1)
self.assertEqual(matches["skills"][0]["title"], "gstack-cli")
self.assertEqual(len(matches["github"]), 1)
self.assertEqual(matches["github"][0]["repo"], "acme/gstack")
class FeaturedPickWorkflowTests(unittest.TestCase):
def test_fallback_without_llm(self):
llm_input = dict(SAMPLE_INPUT)
with patch.dict(os.environ, {"DAILY_FEATURED_PICK": "gstack"}, clear=True):
with patch("daily.featured_pick.has_llm_configured", return_value=False):
featured = research_featured_pick(llm_input, date_str="2026-07-03")
self.assertIsNotNone(featured)
assert featured is not None
self.assertEqual(featured["type"], "skill")
self.assertIn("npx skills add foo/bar/gstack-cli", featured["command"])
self.assertTrue(featured["why_today"])
def test_apply_featured_pick_mutates_input(self):
llm_input = dict(SAMPLE_INPUT)
with patch.dict(os.environ, {"DAILY_FEATURED_PICK": "gstack"}, clear=True):
with patch("daily.featured_pick.has_llm_configured", return_value=False):
featured = apply_featured_pick(llm_input, date_str="2026-07-03")
self.assertIsNotNone(featured)
self.assertIn("featured_pick", llm_input)
self.assertEqual(pick_command_from_featured(featured), llm_input["featured_pick"]["command"])
self.assertEqual(pick_why_from_featured(featured), llm_input["featured_pick"]["why_today"])
if __name__ == "__main__":
unittest.main()

51
tests/test_featured_resolve.py
View File

@@ -1,10 +1,19 @@
# tests/test_featured_resolve.py # tests/test_featured_resolve.py
from __future__ import annotations from __future__ import annotations
import json
import random import random
import tempfile
import unittest import unittest
from pathlib import Path
from unittest.mock import patch
from daily.featured_pick import featured_identity_key, featured_resolve from daily.featured_pick import (
featured_identity_key,
featured_resolve,
load_recent_featured_keys,
load_yesterday_featured_key,
)
class FeaturedResolveTests(unittest.TestCase): class FeaturedResolveTests(unittest.TestCase):
@@ -78,6 +87,46 @@ class FeaturedResolveTests(unittest.TestCase):
"foo/bar", "foo/bar",
) )
def test_load_yesterday_falls_back_to_featured_pick(self):
"""缺 featured_pick_key 时从 featured_pick.url 推导身份，避免连日重复首推。"""
with tempfile.TemporaryDirectory() as tmp:
out = Path(tmp)
payload = {
"data": {
"date": "2026-07-13",
"featured_pick": {
"type": "github",
"title": "headroom",
"url": "https://github.com/headroomlabs-ai/headroom",
},
}
}
(out / "2026-07-13.data.json").write_text(
json.dumps(payload, ensure_ascii=False), encoding="utf-8"
)
with patch("daily.featured_pick.OUTPUT_DIR", out):
key = load_yesterday_featured_key("2026-07-14")
self.assertEqual(key, "headroomlabs-ai/headroom")
def test_load_recent_falls_back_to_featured_pick(self):
with tempfile.TemporaryDirectory() as tmp:
out = Path(tmp)
payload = {
"data": {
"date": "2026-07-13",
"featured_pick": {
"type": "github",
"url": "https://github.com/headroomlabs-ai/headroom",
},
}
}
(out / "2026-07-13.data.json").write_text(
json.dumps(payload, ensure_ascii=False), encoding="utf-8"
)
with patch("daily.featured_pick.OUTPUT_DIR", out):
keys = load_recent_featured_keys("2026-07-14", days=7)
self.assertIn("headroomlabs-ai/headroom", keys)
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()

71
tests/test_github_search.py Normal file
View File

@@ -0,0 +1,71 @@
"""Tests for GitHub Search pagination / deep pool."""
from __future__ import annotations
import os
import unittest
from unittest.mock import MagicMock, patch
class SearchGithubReposPaginationTests(unittest.TestCase):
def test_search_paginates_beyond_first_page_of_30(self):
from daily.github.search import search_github_repos
def make_items(start: int, n: int) -> list[dict]:
return [
{
"full_name": f"org/repo{i}",
"html_url": f"https://github.com/org/repo{i}",
"description": f"desc {i}",
"language": "Python",
"stargazers_count": 1000 - i,
"created_at": "2026-01-01T00:00:00Z",
}
for i in range(start, start + n)
]
responses = [
MagicMock(status_code=200, json=lambda: {"items": make_items(1, 100)}),
MagicMock(status_code=200, json=lambda: {"items": make_items(101, 50)}),
]
client = MagicMock()
client.__enter__.return_value = client
client.__exit__.return_value = False
client.get.side_effect = responses
with patch.dict(os.environ, {"GITHUB_TOKEN": "test-token"}, clear=False):
with patch("daily.github.search.httpx.Client", return_value=client):
with patch("daily.github.search.github_token", return_value="test-token"):
repos = search_github_repos("stars:>50", 120, require_token=True)
self.assertEqual(len(repos), 120)
self.assertEqual(repos[0]["repo"], "org/repo1")
self.assertEqual(repos[119]["repo"], "org/repo120")
self.assertEqual(client.get.call_count, 2)
first_params = client.get.call_args_list[0].kwargs["params"]
self.assertEqual(first_params["per_page"], 100)
self.assertEqual(first_params["page"], 1)
class GithubBoardDeepPoolTests(unittest.TestCase):
def test_board_select_fills_ten_when_deep_pool_has_fresh_repos(self):
from daily.board_select import board_select
recent = {f"old/r{i}" for i in range(1, 33)}
items = [{"repo": f"old/r{i}"} for i in range(1, 31)] + [
{"repo": f"fresh/r{i}"} for i in range(1, 20)
]
selected = board_select(
board="github_trending",
items=items,
recent_keys=recent,
limit=10,
pool_size=100,
kind="github",
)
self.assertEqual(len(selected), 10)
self.assertTrue(all(r["repo"].startswith("fresh/") for r in selected))
if __name__ == "__main__":
unittest.main()

156
tests/test_news_fetch_window.py Normal file
View File

@@ -0,0 +1,156 @@
"""Tests for news time window filtering."""
from __future__ import annotations
import os
import unittest
from datetime import datetime, timezone, timedelta
from unittest.mock import patch
from zoneinfo import ZoneInfo
from daily.news.fetch import _cutoff_datetime, _parse_datetime, _within_window
class NewsWindowTests(unittest.TestCase):
def test_cutoff_floor_today_excludes_yesterday_even_within_24h(self):
tz = ZoneInfo("Asia/Shanghai")
# 2026-07-09 09:00 CST = 2026-07-09 01:00 UTC
fixed = datetime(2026, 7, 9, 1, 0, tzinfo=timezone.utc)
with patch("daily.news.fetch._now_utc", return_value=fixed):
with patch.dict(os.environ, {"DAILY_AI_NEWS_HOURS": "24"}, clear=False):
cutoff = _cutoff_datetime(floor_today=True)
start_today_cst = datetime(2026, 7, 9, 0, 0, tzinfo=tz).astimezone(timezone.utc)
self.assertEqual(cutoff, start_today_cst)
yesterday = datetime(2026, 7, 8, 20, 0, tzinfo=tz).astimezone(timezone.utc)
self.assertFalse(_within_window({"published": yesterday.isoformat()}, cutoff))
def test_cutoff_rolling_only_includes_last_24h(self):
fixed = datetime(2026, 7, 9, 12, 0, tzinfo=timezone.utc)
with patch("daily.news.fetch._now_utc", return_value=fixed):
with patch.dict(os.environ, {"DAILY_AI_NEWS_HOURS": "24"}, clear=False):
cutoff = _cutoff_datetime(floor_today=False)
self.assertEqual(cutoff, fixed - timedelta(hours=24))
def test_within_window_rejects_missing_datetime(self):
cutoff = datetime(2026, 7, 9, 0, 0, tzinfo=timezone.utc)
self.assertFalse(_within_window({"title": "x", "link": "https://a.com"}, cutoff))
def test_parse_date_only_uses_local_noon(self):
with patch.dict(os.environ, {"DAILY_AI_NEWS_TZ": "Asia/Shanghai"}, clear=False):
dt = _parse_datetime("2026-07-09")
self.assertIsNotNone(dt)
assert dt is not None
local = dt.astimezone(ZoneInfo("Asia/Shanghai"))
self.assertEqual(local.hour, 12)
class NewsFormatTests(unittest.TestCase):
def test_ai_news_lines_use_desc_as_link_text(self):
from daily.format_wecom import _ai_news_lines
lines = _ai_news_lines(
[
{
"title": "English Title",
"link": "https://example.com/a",
"source_name": "Src",
"published_fmt": "07-11",
"desc_short": "中文摘要一句",
}
]
)
self.assertEqual(len(lines), 1)
self.assertIn("[中文摘要一句](https://example.com/a)", lines[0])
self.assertNotIn("English Title", lines[0])
self.assertNotIn("> ", lines[0])
def test_ai_news_lines_fallback_to_title(self):
from daily.format_wecom import _ai_news_lines
lines = _ai_news_lines(
[
{
"title": "仅标题",
"link": "https://example.com/b",
"source_name": "Src",
"published_fmt": "",
"desc_short": "",
}
]
)
self.assertIn("[仅标题](https://example.com/b)", lines[0])
class NewsSummaryTests(unittest.TestCase):
def test_brief_news_summary_no_ellipsis(self):
from daily.news.fetch import brief_news_summary
text = (
"Meta told Dylan Byers, of Puck News, that the company removed "
"the controversial AI feature after user backlash on Instagram."
)
out = brief_news_summary(text, limit=72)
self.assertNotIn("...", out)
self.assertLessEqual(len(out), 72)
self.assertTrue(out.startswith("Meta told"))
def test_brief_news_summary_filters_junk(self):
from daily.news.fetch import brief_news_summary
self.assertEqual(brief_news_summary("点击查看原文>"), "")
self.assertEqual(brief_news_summary("Article URL: https://example.com"), "")
def test_sync_wecom_news_rows_after_localize(self):
from daily.news.fetch import _to_wecom_news_row, sync_wecom_news_rows
row = _to_wecom_news_row(
{
"title": "t",
"link": "https://a.com/x",
"source_name": "s",
"published_fmt": "07-11",
"summary": "Short english stub that was truncated early...",
}
)
flat = [
{
"link": "https://a.com/x",
"summary": "苹果指控 OpenAI 窃取硬件商业机密，诉讼称 misconduct 涉及多名前员工。",
}
]
sync_wecom_news_rows([row], flat)
self.assertNotIn("...", row["desc_short"])
self.assertIn("苹果", row["desc_short"])
def test_finalize_wecom_news_forces_chinese(self):
from daily.news.fetch import finalize_wecom_news_items
items = [
{
"link": "https://a.com/1",
"desc_short": "Meta removed the feature after backlash.",
"summary_plain": "Meta removed the feature after backlash.",
}
]
with patch(
"daily.localize.localize_brief_descriptions",
return_value={"wecom-news:https://a.com/1": "Meta 在舆论压力下移除了该功能"},
):
finalize_wecom_news_items(items, force_chinese=True)
self.assertIn("Meta", items[0]["desc_short"])
self.assertNotIn("backlash", items[0]["desc_short"])
class NewsPickTests(unittest.TestCase):
def test_pick_and_backfill_to_limit(self):
from daily.news.fetch import _fill_picked_to_limit, _pick_news_items
flat = [
{"link": f"https://a.com/{i}", "title": f"t{i}", "category_id": "media", "summary": "s"}
for i in range(12)
]
picked = _pick_news_items(flat, 10, ("media",))
self.assertEqual(len(picked), 10)
picked = _fill_picked_to_limit(picked[:3], [flat], 10)
self.assertEqual(len(picked), 10)

118
tests/test_scheduler.py Normal file
View File

@@ -0,0 +1,118 @@
"""Tests for daily.scheduler."""
from __future__ import annotations
import unittest
from datetime import datetime
from zoneinfo import ZoneInfo
from daily.scheduler import (
ClockTime,
SchedulerState,
next_occurrence_after,
parse_hhmm,
plan_next_action,
)
class ParseHhmmTests(unittest.TestCase):
def test_parse(self):
t = parse_hhmm("08:50")
self.assertEqual((t.hour, t.minute), (8, 50))
def test_invalid(self):
with self.assertRaises(ValueError):
parse_hhmm("25:00")
class PlanNextActionTests(unittest.TestCase):
def setUp(self) -> None:
self.tz = ZoneInfo("Asia/Shanghai")
self.gen = ClockTime(8, 50)
self.push = ClockTime(9, 0)
def test_before_generate_waits_for_generate(self):
now = datetime(2026, 7, 9, 8, 30, tzinfo=self.tz)
run_at, action = plan_next_action(
now=now,
tz=self.tz,
state=SchedulerState(),
generate_at=self.gen,
push_at=self.push,
)
self.assertEqual(action, "generate")
self.assertEqual(run_at.hour, 8)
self.assertEqual(run_at.minute, 50)
def test_after_generate_before_push_waits_for_push(self):
now = datetime(2026, 7, 9, 8, 55, tzinfo=self.tz)
state = SchedulerState(last_generate_date="2026-07-09")
run_at, action = plan_next_action(
now=now,
tz=self.tz,
state=state,
generate_at=self.gen,
push_at=self.push,
)
self.assertEqual(action, "push")
self.assertEqual(run_at.hour, 9)
def test_catch_up_generate_when_started_late(self):
now = datetime(2026, 7, 9, 8, 55, tzinfo=self.tz)
run_at, action = plan_next_action(
now=now,
tz=self.tz,
state=SchedulerState(),
generate_at=self.gen,
push_at=self.push,
)
self.assertEqual(action, "generate")
self.assertEqual(run_at, now)
def test_next_day_after_both_done(self):
now = datetime(2026, 7, 9, 10, 0, tzinfo=self.tz)
state = SchedulerState(last_generate_date="2026-07-09", last_push_date="2026-07-09")
run_at, action = plan_next_action(
now=now,
tz=self.tz,
state=state,
generate_at=self.gen,
push_at=self.push,
)
self.assertEqual(action, "generate")
self.assertEqual(run_at.date().isoformat(), "2026-07-10")
def test_evening_start_waits_for_tomorrow_generate(self):
now = datetime(2026, 7, 9, 20, 35, tzinfo=self.tz)
run_at, action = plan_next_action(
now=now,
tz=self.tz,
state=SchedulerState(),
generate_at=self.gen,
push_at=self.push,
)
self.assertEqual(action, "generate")
self.assertEqual(run_at.date().isoformat(), "2026-07-10")
self.assertEqual((run_at.hour, run_at.minute), (8, 50))
def test_catch_up_push_when_generate_done(self):
now = datetime(2026, 7, 9, 20, 35, tzinfo=self.tz)
state = SchedulerState(last_generate_date="2026-07-09")
run_at, action = plan_next_action(
now=now,
tz=self.tz,
state=state,
generate_at=self.gen,
push_at=self.push,
)
self.assertEqual(action, "push")
self.assertEqual(run_at, now)
class NextOccurrenceTests(unittest.TestCase):
def test_tomorrow_when_past(self):
tz = ZoneInfo("Asia/Shanghai")
now = datetime(2026, 7, 9, 10, 0, tzinfo=tz)
nxt = next_occurrence_after(ClockTime(8, 50), tz, now)
self.assertEqual(nxt.date().isoformat(), "2026-07-10")
self.assertEqual((nxt.hour, nxt.minute), (8, 50))

279
tests/test_wecom_delta.py
View File

@@ -229,6 +229,192 @@ class DeltaFormatTests(unittest.TestCase):
self.assertIn("Skills Trending Top 10", text) self.assertIn("Skills Trending Top 10", text)
self.assertNotIn("Skills Trending 变化", text) self.assertNotIn("Skills Trending 变化", text)
def test_delta_pad_keeps_large_clusters_merged_and_fills_limit(self):
"""补榜按 source 合并态取条，大 cluster 不得撑爆 flat 预算导致短榜。"""
from daily.format_wecom import build_skills_delta_sections
def cluster(source: str, n: int, installs: int) -> dict:
titles = [f"t{i}" for i in range(n)]
return {
"id": f"{source}/{titles[0]}",
"title": titles[0],
"source": source,
"installs": installs,
"installs_fmt": str(installs),
"cluster": True,
"cluster_count": n,
"cluster_skills": titles,
"cluster_titles": ", ".join(titles[:4]) + "…",
"link": f"https://skills.sh/{source}/{titles[0]}",
"description": f"{source} cluster",
}
full = [cluster(f"big{i}/pkg", 20, 1000 - i) for i in range(1, 5)] + [
{
"id": f"other{n}/pkg/skill",
"title": "skill",
"source": f"other{n}/pkg",
"installs": 50 - n,
"installs_fmt": str(50 - n),
"link": f"https://skills.sh/other{n}/pkg/skill",
"description": f"other {n}",
}
for n in range(1, 12)
]
with patch("daily.format_wecom.localize_brief_descriptions", return_value={}):
with patch("daily.format_wecom.needs_chinese", return_value=False):
text = build_skills_delta_sections(
[],
[],
trending_full=full,
hot_full=[],
trending_limit=10,
pad=True,
)
self.assertIn("Skills Trending Top 10", text)
self.assertIn("20 skills", text)
self.assertIn("other6/pkg", text)
def test_delta_pad_recent_blocks_same_source_not_just_primary_id(self):
"""周去重按 source：换同仓另一个 skill id 不得再上榜。"""
from daily.format_wecom import build_skills_delta_sections
titles = [f"t{i}" for i in range(20)]
full = [
{
"id": f"big/pkg/{titles[0]}",
"title": titles[0],
"source": "big/pkg",
"installs": 999,
"installs_fmt": "999",
"cluster": True,
"cluster_count": 20,
"cluster_skills": titles,
"cluster_titles": ", ".join(titles[:4]) + "…",
"link": f"https://skills.sh/big/pkg/{titles[0]}",
"description": "big cluster",
},
*[
{
"id": f"other{n}/pkg/skill",
"title": "skill",
"source": f"other{n}/pkg",
"installs": 50 - n,
"installs_fmt": str(50 - n),
"link": f"https://skills.sh/other{n}/pkg/skill",
"description": f"other {n}",
}
for n in range(1, 12)
],
]
# 昨日展示的是同 source 另一 skill id（非今日 primary）
recent = {f"big/pkg/{titles[5]}"}
with patch("daily.format_wecom.localize_brief_descriptions", return_value={}):
with patch("daily.format_wecom.needs_chinese", return_value=False):
text = build_skills_delta_sections(
[],
[],
trending_full=full,
hot_full=[],
trending_limit=10,
pad=True,
recent_trending=recent,
)
self.assertIn("Skills Trending Top 10", text)
self.assertNotIn("big/pkg", text)
self.assertIn("other1/pkg", text)
def test_delta_pad_hot_recent_unions_trending_history_by_source(self):
"""Skills Hot 周去重合并 Trending 历史：隔日换榜也不能同 source 再出现。"""
from daily.format_wecom import build_skills_delta_sections
hot_full = [
{
"id": "101-skills/skills/ai-music",
"title": "ai-music",
"source": "101-skills/skills",
"installs": 200,
"installs_fmt": "200",
"link": "https://skills.sh/101-skills/skills/ai-music",
"description": "hot candidate",
},
{
"id": "fresh/src/skill",
"title": "skill",
"source": "fresh/src",
"installs": 100,
"installs_fmt": "100",
"link": "https://skills.sh/fresh/src/skill",
"description": "fresh",
},
]
with patch("daily.format_wecom.localize_brief_descriptions", return_value={}):
with patch("daily.format_wecom.needs_chinese", return_value=False):
text = build_skills_delta_sections(
[],
[],
trending_full=[],
hot_full=hot_full,
trending_limit=10,
hot_limit=10,
pad=True,
recent_trending={"101-skills/skills/ai-video-generation"},
recent_hot=set(),
)
self.assertIn("Skills Hot Top 1", text)
self.assertIn("fresh/src", text)
self.assertNotIn("101-skills", text)
def test_delta_pad_hot_excludes_trending_by_source(self):
"""同日 Hot 补榜按 source 避开 Trending，而非展开全部 cluster skill id。"""
from daily.format_wecom import build_skills_delta_sections
trending_full = [
{
"id": "same/src/a",
"title": "a",
"source": "same/src",
"installs": 100,
"installs_fmt": "100",
"link": "https://skills.sh/same/src/a",
"description": "trending item",
}
]
hot_full = [
{
"id": "same/src/b",
"title": "b",
"source": "same/src",
"installs": 90,
"installs_fmt": "90",
"link": "https://skills.sh/same/src/b",
"description": "hot twin",
},
{
"id": "fresh/src/skill",
"title": "skill",
"source": "fresh/src",
"installs": 80,
"installs_fmt": "80",
"link": "https://skills.sh/fresh/src/skill",
"description": "fresh hot",
},
]
with patch("daily.format_wecom.localize_brief_descriptions", return_value={}):
with patch("daily.format_wecom.needs_chinese", return_value=False):
text = build_skills_delta_sections(
[],
[],
trending_full=trending_full,
hot_full=hot_full,
trending_limit=10,
hot_limit=10,
pad=True,
)
self.assertIn("Skills Hot Top 1", text)
self.assertIn("fresh/src", text)
self.assertNotIn("same/src/b", text)
def test_delta_pad_uses_large_pool_when_recent_excludes_top(self): def test_delta_pad_uses_large_pool_when_recent_excludes_top(self):
from daily.format_wecom import build_skills_delta_sections from daily.format_wecom import build_skills_delta_sections
@@ -306,6 +492,46 @@ class DeltaFormatTests(unittest.TestCase):
self.assertIn("GitHub Trending Top 10", text) self.assertIn("GitHub Trending Top 10", text)
self.assertNotIn("GitHub Trending 变化", text) self.assertNotIn("GitHub Trending 变化", text)
def test_delta_pad_github_unions_recent_across_boards(self):
"""GitHub 三榜共用周去重：Trending 出过的 repo，新兴/Topic 不得再出。"""
from daily.format_wecom import build_github_delta_sections
movement = {"github_trending_moves": [], "github_emerging_moves": [], "github_topic_moves": []}
shared = {
"repo": "seen/repo",
"url": "https://github.com/seen/repo",
"language": "Go",
"stars_today_fmt": "100",
"total_stars_fmt": "1K",
"created_at": "2026-07-01",
"description": "already shown",
"desc_short": "already shown",
}
fresh = {
"repo": "fresh/repo",
"url": "https://github.com/fresh/repo",
"language": "Go",
"stars_today_fmt": "90",
"total_stars_fmt": "900",
"created_at": "2026-07-02",
"description": "fresh",
"desc_short": "fresh",
}
with patch("daily.format_wecom.localize_brief_descriptions", return_value={}):
text = build_github_delta_sections(
movement,
topic_name="llm",
github_trending=[],
github_emerging=[shared, fresh],
github_topic=[shared],
emerging_limit=5,
topic_limit=5,
pad=True,
recent_board_keys={"github_trending": {"seen/repo"}},
)
self.assertIn("fresh/repo", text)
self.assertNotIn("seen/repo", text)
def test_delta_pad_skips_recent_skills(self): def test_delta_pad_skips_recent_skills(self):
from daily.format_wecom import build_skills_delta_sections from daily.format_wecom import build_skills_delta_sections
@@ -320,6 +546,16 @@ class DeltaFormatTests(unittest.TestCase):
"description": f"skill {i}", "description": f"skill {i}",
} }
for i in range(4, 6) for i in range(4, 6)
] + [
{
"id": "fresh/src/skill",
"title": "skill",
"source": "fresh/src",
"installs": 50,
"installs_fmt": "50",
"link": "https://skills.sh/fresh/src/skill",
"description": "fresh skill",
}
] ]
with patch("daily.format_wecom.localize_brief_descriptions", return_value={}): with patch("daily.format_wecom.localize_brief_descriptions", return_value={}):
with patch("daily.format_wecom.needs_chinese", return_value=False): with patch("daily.format_wecom.needs_chinese", return_value=False):
@@ -330,10 +566,12 @@ class DeltaFormatTests(unittest.TestCase):
hot_full=[], hot_full=[],
trending_limit=10, trending_limit=10,
pad=True, pad=True,
# 同仓历史 skill id → 整仓 source 去重；仅保留其它 source
recent_trending={f"x/y/s{i}" for i in range(1, 4)}, recent_trending={f"x/y/s{i}" for i in range(1, 4)},
) )
self.assertIn("Skills Trending Top 1", text) self.assertIn("Skills Trending Top 1", text)
self.assertIn("2 skills", text) self.assertIn("fresh/src", text)
self.assertNotIn("x/y", text)
def test_delta_pad_skips_recent_github(self): def test_delta_pad_skips_recent_github(self):
from daily.format_wecom import build_github_delta_sections from daily.format_wecom import build_github_delta_sections
@@ -363,6 +601,45 @@ class DeltaFormatTests(unittest.TestCase):
self.assertIn("org/r4", text) self.assertIn("org/r4", text)
self.assertNotIn("org/r1", text) self.assertNotIn("org/r1", text)
def test_delta_pad_skips_recent_github_moves(self):
"""异动新入榜若昨日企微已展示，pad 时仍应排除（不只滤补榜）。"""
from daily.format_wecom import build_github_delta_sections
movement = {
"github_trending_moves": [
{
"repo": "vinta/awesome-python",
"url": "https://github.com/vinta/awesome-python",
"language": "Python",
"total_stars_fmt": "308K",
"description": "list",
}
],
"github_emerging_moves": [],
"github_topic_moves": [],
}
full = [
{
"repo": "fresh/repo",
"url": "https://github.com/fresh/repo",
"language": "Go",
"total_stars_fmt": "1K",
"description": "fresh",
"desc_short": "fresh",
}
]
with patch("daily.format_wecom.localize_brief_descriptions", return_value={}):
text = build_github_delta_sections(
movement,
topic_name="llm",
github_trending=full,
trending_limit=10,
pad=True,
recent_board_keys={"github_trending": {"vinta/awesome-python"}},
)
self.assertIn("fresh/repo", text)
self.assertNotIn("vinta/awesome-python", text)
def test_load_recent_board_keys_from_data_json(self): def test_load_recent_board_keys_from_data_json(self):
import json import json
import tempfile import tempfile