Compare commits

13 Commits

Author SHA1 Message Date
db270013eb fix: retry Agent Step2 once when first write attempt fails
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 16:40:05 +08:00
64179820d7 fix: retry transient RSS fetch failures with backoff
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 16:39:11 +08:00
f6fac7b642 fix: use browser User-Agent for RSS feeds blocked by InfoQ
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 16:38:48 +08:00
37931d2ef5 fix: raise Cursor SDK bridge timeout for daily LLM calls
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 16:38:21 +08:00
3936467838 删除 git actions 的相关配置 2026-07-03 15:30:41 +08:00
5742dc47ed feat: LLM 解耦 bot、RSS 外置与内容过滤
daily 自建 Cursor bridge;DAILY_LLM_PROVIDER 控制后端;config/feeds.yaml 与 sensitive_words 可配置。

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 15:24:58 +08:00
ce04d5a342 refactor: Phase 3 拆分 generate 流水线并补全测试
Some checks failed
test / pytest (push) Failing after 3s
daily / report (push) Failing after 1s
提取 collect/formatters/themes 等 pipeline 模块,新增 wecom 分条、RSS、delta、Agent 工作流测试。

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 14:59:12 +08:00
391f887d73 feat: Phase 2 新闻打分去重与 Agent 输入池裁剪
标题相似度合并、信源/时效/昨日重复加权排序,Agent 模式扩大 LLM 新闻候选池并记录抓取失败源。

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 14:53:30 +08:00
728bd1b9d1 feat: Phase 1 添加 GHA 定时、run-daily.sh 与 Docker 部署
让早报可在无本地服务器环境下定时生成并推送企微,补齐 Linux/CI 入口。

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 14:51:11 +08:00
8075ab78e8 test: Phase 3 添加 smoke test、CI 与 run-daily 去重锁
离线 mock 测试 generate 产出,GitHub Actions 跑 pytest,30 分钟内重复调度自动 skip。

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 14:45:58 +08:00
ebc25c9e9b docs: Phase 2 重写 onboarding 文档与示例产出
主 README 聚焦早报 Quick Start,Bot 文档迁至 bot/README.md 并标 experimental。

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 14:45:48 +08:00
f533e31adb refactor: Phase 1 抽出 shared/skills_data 解耦 daily 与 bot
daily 不再通过 sys.path 导入 bot/skills_service,skills feed 数据层上移到 shared/。

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 14:45:39 +08:00
88d08c0da9 chore: Phase 0 统一命名与仓库卫生
统一 daily-robots 命名、修正示例路径与 User-Agent,并清理 tmp 与 IDE 追踪文件。

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 14:45:19 +08:00
62 changed files with 2980 additions and 1286 deletions

18
.dockerignore Normal file
View File

@@ -0,0 +1,18 @@
.git
.github
.env
.env.local
output
.cache
logs
bot/.env
bot/.venv
bot/.cache
__pycache__
*.pyc
.idea
tmp_*
tests
.pytest_cache
*.md
!skills/**/*.md

View File

@@ -59,9 +59,14 @@ DAILY_CN_AI_NEWS=1
# Agent 工作流(推荐:叙事化早报)
# DAILY_REPORT_MODE=agent
# DAILY_LLM_PROVIDER=auto
# auto — agent 模式优先 Cursor否则 OpenAIclassic 优先 OpenAI
# openai — 强制 OpenAI 兼容 API
# cursor — 强制 Cursor SDK
# CURSOR_API_KEY=cursor_...
# CURSOR_MODEL=composer-2.5
# DAILY_CURSOR_CWD=d:\LY\diy\skills-hot-daily
# CURSOR_TIMEOUT=600
# DAILY_CURSOR_CWD=.
# 新增榜对比(较昨日 Top15供 Agent 导语/signals列表展示 Top N
# DAILY_DELTA_COMPARE_DEPTH=15
@@ -73,6 +78,19 @@ DAILY_AI_NEWS_HOURS=72
DAILY_AI_NEWS_PER_FEED=3
DAILY_AI_NEWS_PER_CATEGORY=5
# 新闻排序 / 去重Phase 2
# 标题 Jaccard 相似度阈值0-95默认 55 表示 0.55
# DAILY_NEWS_TITLE_SIM=55
# Agent 模式 LLM 输入池classic 仍用 DAILY_WECOM_* 条数)
# DAILY_AGENT_NEWS_POOL=40
# DAILY_AGENT_CN_NEWS_POOL=30
# RSS 源:编辑 config/feeds.yamlintl / cn缺失时回退内置默认
# 单源失败重试次数(默认 3含首次请求
# DAILY_RSS_RETRY=3
# 内容过滤config/sensitive_words.yaml + DAILY_CONTENT_FILTER=1
# DAILY_CONTENT_FILTER=0
# Reddit RSS403/429 时在 Reddit 偏好设置 → RSS feeds 复制 user / feed 参数)
# REDDIT_RSS_USER=your_username
# REDDIT_RSS_FEED=your_feed_token

6
.gitignore vendored
View File

@@ -5,4 +5,8 @@ logs/
bot/.env
bot/.venv/
bot/.cache/
output
output
__pycache__/
*.pyc
.idea/
tmp_*

3
.idea/.gitignore generated vendored
View File

@@ -1,3 +0,0 @@
# Default ignored files
/shelf/
/workspace.xml

View File

@@ -1,9 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<module type="JAVA_MODULE" version="4">
<component name="NewModuleRootManager" inherit-compiler-output="true">
<exclude-output />
<content url="file://$MODULE_DIR$" />
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
</component>
</module>

2
.idea/misc.xml generated
View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ProjectRootManager" version="2" languageLevel="JDK_21" default="true" project-jdk-name="21" project-jdk-type="JavaSDK">
<component name="ProjectRootManager" version="2" default="true">
<output url="file://$PROJECT_DIR$/out" />
</component>
</project>

8
.idea/modules.xml generated
View File

@@ -1,8 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ProjectModuleManager">
<modules>
<module fileurl="file://$PROJECT_DIR$/.idea/daily-robots.iml" filepath="$PROJECT_DIR$/.idea/daily-robots.iml" />
</modules>
</component>
</project>

2
.idea/vcs.xml generated
View File

@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="VcsDirectoryMappings">
<mapping directory="" vcs="Git" />
<mapping directory="$PROJECT_DIR$" vcs="Git" />
</component>
</project>

27
Dockerfile Normal file
View File

@@ -0,0 +1,27 @@
FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \
TZ=Asia/Shanghai
WORKDIR /app
RUN apt-get update \
&& apt-get install -y --no-install-recommends tzdata \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY daily/ daily/
COPY shared/ shared/
COPY skills/ skills/
COPY config/ config/
COPY run-daily.sh .
RUN chmod +x run-daily.sh \
&& mkdir -p output .cache logs
VOLUME ["/app/output", "/app/.cache", "/app/logs"]
ENTRYPOINT ["./run-daily.sh"]

465
README.md
View File

@@ -1,338 +1,149 @@
# skills-hot-daily
# daily-robots
Skills / GitHub 早报推送 + 企微对话机器人(同一仓库、两套企微接入)。
> **Daily Briefing** · **WeCom Push** · **Agent Mode**
给 AI 工程小团队推送**每日 curated 情报**skills.sh 榜单、GitHub AI 趋势、国际/国内 RSS生成叙事化早报并推到企业微信群。
与 TLDR AI / Ben's Bites 不同,这里专注 **Agent Skills 生态** + **可 fork 的自托管流水线**(不是又一个 email newsletter
**产出示例**[docs/sample-output/wecom-agent-sample.md](docs/sample-output/wecom-agent-sample.md)Agent 模式企微版节选)
---
## Quick Start
1. **Clone** 本仓库
2. **安装依赖**`pip install -r requirements.txt`
3. **配置环境**`copy .env.example .env`
4. **填写 webhook**:企微群 → 群机器人 → 添加 → 把 `key=` 写入 `.env`
```env
WECOM_WEBHOOK_KEY=your-webhook-key
```
5. **生成并推送**`.\run-daily.ps1`
生成文件在 `output/`
| 文件 | 说明 |
|------|------|
| `YYYY-MM-DD.wecom.md` | 企微推送版(主产物) |
| `YYYY-MM-DD.md` | 完整归档版 |
| `YYYY-MM-DD.data.json` | 结构化数据(供 LLM / 调试) |
常用变体:
```powershell
.\run-daily.ps1 -SkipPush # 只生成
python -m daily push output\2026-07-03.wecom.md # 只推送
```
Linux / macOS 等价脚本:
```bash
chmod +x run-daily.sh
./run-daily.sh # 生成 + 推送
./run-daily.sh --skip-push # 只生成
./run-daily.sh --force # 忽略 30 分钟内重复运行锁
```
**前置条件**Python 3.10+、企业微信群机器人 webhook。无需 Bot API 凭证。
---
## 部署Docker
```bash
cp .env.example .env # 填入 WECOM_WEBHOOK_KEY 等
docker compose build
docker compose run --rm daily # 生成 + 推送
docker compose run --rm daily --skip-push # 只生成
```
`output/`、`.cache/`、`logs/` 挂载到宿主机,与本地 `run-daily.ps1` 行为一致。
---
## Agent 模式(推荐)
默认 `classic` 是分区榜单模板;**推荐 Agent 模式**——导语 + 今日信号 + 精选新闻 + 榜单,全中文叙述。
```env
DAILY_REPORT_MODE=agent
CURSOR_API_KEY=cursor_...
CURSOR_MODEL=composer-2.5
DAILY_CURSOR_CWD=.
```
| 模式 | 环境变量 | 风格 |
|------|----------|------|
| `classic` | — | 分区榜单 + 模板 |
| `editor` | `DAILY_CURSOR_EDITOR=1` | 模板 + LLM 中文化 |
| **`agent`** | `DAILY_REPORT_MODE=agent` | **叙事化早报(推荐)** |
流水线:
```
Python 抓取 → Step1 趋势分析 (.trends.json) → Step2 写企微稿 (.wecom.md) → webhook 分条推送
```
- 规范见 `skills/daily-agent/SKILL.md`
- LLM 失败自动回退 `classic`,不影响 `run-daily.ps1`
- 无 `CURSOR_API_KEY` 时请用 `classic`,或配置 `DAILY_LLM_API_KEY`OpenAI 兼容 API
定时推送Windows 任务计划程序每日执行 `run-daily.ps1`,或 cron / `schtasks`。
---
## 自定义 RSS 源
RSS 源配置在 **`config/feeds.yaml`**(国际 `intl` / 国内 `cn`),改 URL 或增删 feed 后无需改 Python。文件缺失或解析失败时会回退 `daily/news/feeds_defaults.py` 内置列表。
可选 **`config/sensitive_words.yaml`** + `DAILY_CONTENT_FILTER=1`:标题/摘要命中敏感词则丢弃该条。
旧版硬编码路径(仍可读作参考):`daily/news/feeds_defaults.py`
```env
DAILY_AI_NEWS=1
DAILY_CN_AI_NEWS=1
DAILY_AI_NEWS_HOURS=72
DAILY_WECOM_AI_NEWS=10
DAILY_WECOM_CN_AI_NEWS=8
```
Skills 榜单默认爬 skills.sh 官网600+ 条);可改为第三方 feed
```env
# SKILLS_BOARD_SOURCE=feed
```
GitHub 新兴/Topic 榜需 `GITHUB_TOKEN`(见 `.env.example` 注释)。
---
## Experimental`bot/` 企微对话机器人
同仓库内的 **API 模式智能机器人**@ 机器人快查 skills、Cursor 任务、Playwright 截图)**不在 v1 开源支持范围内**,需独立 venv 与 `WECOM_BOT_ID` / `SECRET`。
详见 **[bot/README.md](bot/README.md)**。
---
## 项目结构
```
skills-hot-daily/
├── README.md
├── .env.example # 早报 webhook、GitHub 等
├── requirements.txt # 早报 Python 依赖
├── run-daily.ps1 # 生成 + 推送一条龙
├── send-wecom.ps1 # 仅推送
├── daily/ # 早报 Python 包
│ ├── __main__.py # python -m daily [generate|push]
│ ├── config.py
│ ├── generate.py
│ ├── report_data.py # JSON 中间层
│ ├── cursor_editor.py # Cursor 编辑层
│ ├── llm_client.py
│ ├── localize.py # 仅中文化Tier A
│ ├── format_wecom.py
│ ├── webhook.py
│ ├── news/ # 国际 AI 时讯 RSS
│ │ ├── feeds.py
│ │ └── fetch.py
│ └── github/
│ ├── auth.py
│ ├── search.py
│ └── trending.py
├── output/ # YYYY-MM-DD.md / .wecom.md / .data.json / .editorial.json
├── skills/daily-editor/ # 早报 Cursor 编辑 Skill
│ └── SKILL.md
├── logs/
├── .cache/
└── bot/ # 企微 API 模式对话机器人(独立 venv
├── main.py
├── skills_service.py
└── scenarios/
daily-robots/
├── run-daily.ps1 # 生成 + 推送
├── daily/ # 早报主包
├── shared/skills_data.py # skills feed 共用数据层
├── skills/daily-agent/ # Agent 工作流规范
├── output/ # 生成产物gitignore
└── bot/ # experimental
```
| 模块 | 配置文件 | 启动方式 |
|------|----------|----------|
| **早报推送** | 根目录 `.env``WECOM_WEBHOOK_KEY` 等) | `.\run-daily.ps1` |
| **对话 Bot** | `bot/.env``WECOM_BOT_ID` / `SECRET` 等) | `cd bot``python main.py` |
---
## 一、早报推送
```powershell
cd d:\LY\test\tech\skills-hot-daily
pip install -r requirements.txt
copy .env.example .env
.\run-daily.ps1
```
- 生成:`output/YYYY-MM-DD.md`(完整版)、`output/YYYY-MM-DD.wecom.md`(企微短版)
- 仅生成:`.\run-daily.ps1 -SkipPush`
- 仅推送:`python -m daily push output\2026-06-25.wecom.md`
**Webhook 配置**:企微群 → 群机器人 → 添加,将 `key=` 后的值写入项目根 `.env`
```env
WECOM_WEBHOOK_KEY=your-key
```
**GitHub 数据源**(见 `.env.example`
| 来源 | 说明 |
|------|------|
| GitHub Trending | `GITHUB_TRENDING_MODE=scrape``api` |
| 新兴 / Topic | Search API`GITHUB_TOKEN` |
| Release | 可选 `GITHUB_REPOS=owner/repo` |
**国际 AI 时讯**RSS`daily/news/feeds.py`
| 类别 | 覆盖 |
|------|------|
| 厂商官方 | Anthropic、OpenAI、Google、Meta、Microsoft、Mistral、Cursor 等 |
| Agent / LLM 开发者 | LangChain、LlamaIndex、Hugging Face、Copilot 等 |
| 综合科技媒体 | The Verge、TechCrunch、Ars、Wired、MIT TR 等 |
| Newsletter | Ben's Bites、Latent Space、Simon Willison、TLDR AI 等 |
| 研究 / 论文 | arXiv cs.CL/AI/LG、HF Papers |
| 社区讨论 | HN、Reddit r/LocalLLaMA / ClaudeAI / ML 等 |
环境变量:`DAILY_AI_NEWS=1` · `DAILY_CN_AI_NEWS=1` · `DAILY_AI_NEWS_HOURS=72` · `DAILY_WECOM_AI_NEWS=10` · `DAILY_WECOM_CN_AI_NEWS=8`
**国内 AI 时讯**RSS`daily/news/feeds_cn.py`
| 类别 | 覆盖 |
|------|------|
| AI 专业媒体 | 量子位、InfoQ 中文 |
| 综合科技 | 36氪、雷锋网、Google News 中文 |
| 开发者社区 | 掘金(标题 AI 关键词过滤) |
### 生成架构Tier B · Cursor 编辑层)
早报默认走 **Python 抓取 + 模板渲染**;可选开启 Cursor 做「编辑」:
```
抓取数据 → output/日期.data.json → Cursor 读 Skill 写 editorial → 模板填字 → .md / .wecom.md
```
| 文件 | 说明 |
|------|------|
| `output/YYYY-MM-DD.data.json` | 结构化榜单(供 LLM 输入) |
| `output/YYYY-MM-DD.editorial.json` | Cursor 输出的主题、速览、中文描述 |
| `skills/daily-editor/SKILL.md` | 编辑规范语气、JSON 格式) |
```env
# 开启 Tier B需 CURSOR_API_KEY 或 DAILY_LLM_API_KEY
DAILY_CURSOR_EDITOR=1
```
- 开启后:**一次 LLM 调用** 生成 `theme_line` + `highlights` + 全部中文描述
- 关闭时(默认):规则主题 + `DAILY_ZH_DESC` 仅中文化描述
- LLM 失败自动回退规则模式,不影响推送
### 生成架构Agent 工作流 · 推荐)
若觉得模板版「榜单堆砌」不友好,可改用 **Agent 三步流水线**
```
Python 抓取 → Step1 趋势分析 → Step2 叙事写稿 → Python 分条推送企微
(.trends.json) (.wecom.md)
```
| 模式 | 环境变量 | 企微版风格 |
|------|----------|------------|
| `classic`(默认) | — | 分区榜单 + 模板 |
| `editor` | `DAILY_CURSOR_EDITOR=1` | 模板 + LLM 中文化 |
| `agent` | `DAILY_REPORT_MODE=agent` | **导语 + 信号 + 精选**,全中文叙述 |
```env
DAILY_REPORT_MODE=agent
DAILY_CURSOR_CWD=d:\LY\diy\skills-hot-daily # 早报 LLM 工作目录(与 bot 的 CURSOR_CWD 独立)
```
| 文件 | 说明 |
|------|------|
| `output/YYYY-MM-DD.trends.json` | Step1 趋势分析结果 |
| `skills/daily-agent/SKILL.md` | Agent 工作流规范 |
- 完整版 `YYYY-MM-DD.md` 仍为数据表格归档;企微版由 Agent 直接写 Markdown
- Agent 失败自动回退 `classic`,不影响 `run-daily.ps1`
定时推送Windows 任务计划程序或 `/loop 1d` 执行 `run-daily.ps1`
---
## 二、可对话 Skills 助手(企业微信智能机器人)
在企微里 @ 机器人即可:
- **快查**`trending 10``hot 10``搜索 react`(本地 skills 数据,秒回)
- **截图预览**`preview` / `截图`(基于 `.env``CURSOR_CWD` 启动前端并发图)
- **通用任务**:任意自然语言需求,由 **Cursor Agent** 执行并回传结果
### 1. 创建 API 模式机器人
1. [企业微信管理后台](https://work.weixin.qq.com/) → **安全与管理****管理工具****智能机器人****创建机器人**
2. 选择 **API 模式创建****使用长连接**
3. 记录 **Bot ID****Secret**Secret 只显示一次,请立即保存)
4. 设置可见范围,将机器人 **添加到目标群** 或允许成员单聊
普通成员路径:工作台 → 智能机器人 → 手动创建 → API 模式 → 长连接
### 2. 启动本地服务
```powershell
cd d:\LY\test\tech\skills-hot-daily\bot
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
playwright install chromium
copy .env.example .env
# 编辑 .envWECOM_BOT_ID / WECOM_BOT_SECRET / CURSOR_API_KEY
python main.py
```
服务需 **常驻运行**(本机、服务器或 Docker。长连接模式下机器人进程须在线才能收消息。
### 3. 路由模式ROUTING_MODE
| 模式 | 行为 |
|------|------|
| `hybrid`(默认) | `trending`/`hot`/`搜索`/`详情` 走本地快查;其余 @ 消息交给 Cursor |
| `cursor` | 所有消息都交给 Cursor 执行 |
| `skills` | 仅本地 skills 快查(旧行为) |
**Cursor 任务示例**(群里发送):
```
@test 总结 trending top10并推荐 3 个适合前端团队的 skill
@test 对比 mattpocock/skills 和 obra/superpowers 各有哪些热门 skill
@test 帮我写一段 npx skills add 的安装说明
```
Cursor 在本机 `CURSOR_CWD` 目录下运行,默认 `d:\LY\test\tech`。复杂任务可能需要 110 分钟流式消息会显示「Cursor 正在执行任务…」。
### 4. 前端截图预览API 模式发图)
项目路径读取 `.env` 中的 **`CURSOR_CWD`**。机器人会:
1.`CURSOR_CWD` 检测 `package.json`,若有 `dev` 脚本则执行 `PREVIEW_DEV_COMMAND`(默认 `npm run dev`
2. 等待 `PREVIEW_PORT`(默认 `5173`)就绪,或用 `PREVIEW_URL` 直接访问
3. Playwright 打开页面并截图
4. 通过 API 模式 **上传图片 + 回复 image 消息** 到群
| 命令 | 说明 |
|------|------|
| `preview` / `截图` / `预览` | 访问 `http://127.0.0.1:5173/` 并截图 |
| `preview /login` | 指定路径 |
| `preview / 3000` | 指定端口 |
| `preview http://127.0.0.1:8080/` | 指定完整 URL |
**多步网页操作**(登录、点菜单、再截图)见下一节,不再写死在代码里。
`.env` 可选配置:
```env
CURSOR_CWD=d:\LY\test\tech
PREVIEW_PORT=5173
PREVIEW_URL=http://127.0.0.1:5173/
PREVIEW_DEV_COMMAND=npm run dev
PREVIEW_STARTUP_TIMEOUT=120
```
`CURSOR_CWD` 下暂无前端项目,可先手动启动 dev server或设置 `PREVIEW_URL` 指向已运行地址。
### 4b. 网页操作Playwright 步骤引擎)
支持三种方式定义操作流程,**无需改 Python 代码**
**1. 自然语言(企微里直接说)**
```
@test 访问登录页,输入账号密码,点击登录后进入主页,点击智能体管理菜单然后截图
```
账号密码从 `.env` 读取(`{{PREVIEW_LOGIN_USER}}` / `{{PREVIEW_LOGIN_PASSWORD}}`),勿在群里发密码。
**2. 场景文件 YAML**
`bot/scenarios/xiaobao-agent-manage.yaml` 示例:
```yaml
name: xiaobao-agent-manage
steps:
- goto: /login
- fill:
field: 账号
value: "{{PREVIEW_LOGIN_USER}}"
- fill:
field: 密码
value: "{{PREVIEW_LOGIN_PASSWORD}}"
- click: 登录
- wait:
url: "**/app/**"
- click: 智能体管理
- wait: 1500
- screenshot
```
触发:`@test browser xiaobao-agent-manage`
场景搜索路径:`bot/scenarios/``CURSOR_CWD/.browser-scenarios/`、环境变量 `BROWSER_SCENARIOS_DIR`
**3. 消息内 DSL**
```
browser:
goto /login
fill 账号 {{PREVIEW_LOGIN_USER}}
fill 密码 {{PREVIEW_LOGIN_PASSWORD}}
click 登录
click 智能体管理
screenshot
```
**支持的步骤**`goto` · `fill` · `click` · `wait` · `screenshot` · `press`
`.env` 登录与场景配置:
```env
PREVIEW_LOGIN_USER=test_account
PREVIEW_LOGIN_PASSWORD=your_password
# BROWSER_DEFAULT_SCENARIO=xiaobao-agent-manage
```
### 5. 支持的快查命令
| 命令 | 说明 |
|------|------|
| `trending 10` / `趋势 10` | 近期增长榜 Top N默认 10最大 30 |
| `hot 10` / `实时 10` | 实时热度榜 |
| `all 10` / `总榜 10` | 历史总安装榜 |
| `搜索 react` / `search tdd` | 关键词搜索 |
| `详情 find-skills` | 单个 skill 详情 + 安装命令 |
| `preview` / `截图` | 启动 CURSOR_CWD 前端并截图发群 |
| `帮助` | 命令列表 |
自然语言(非显式快查命令)会交给 **Cursor** 处理,例如 `@test 查 trending 并写推荐`
### 6. 本地测试(无需企微凭证)
```powershell
cd d:\LY\test\tech\skills-hot-daily\bot
python -c "from skills_service import handle_command; print(handle_command('trending 5'))"
```
---
## 其他推送方案
| 方案 | 适用场景 | 复杂度 |
|------|----------|--------|
| **群机器人 webhook** | 推送到固定群 | 低 |
| **应用消息 API** | 推送给指定成员/部门 | 中(需 corp_id、secret、agent_id |
| **邮件 + 企业微信邮箱** | 已有 SMTP | 中 |
| **PushPlus / Server酱** | 个人微信中转 | 低(第三方) |
### 应用消息 API简要
适合「推送给某个人」而非群聊。需在 [企业微信管理后台](https://work.weixin.qq.com/) 创建自建应用,调用:
`POST https://qyapi.weixin.qq.com/cgi-bin/message/send?access_token=TOKEN`
消息体支持 `text` / `markdown` / `news` 等。需先 `gettoken` 再发消息,并维护 access_token 缓存。
---
## 注意事项
- Webhook **不要提交 Git**,只用环境变量
- 企业微信 markdown 为**子集**(不支持完整 GitHub 表格语法时可改为文本列表
- 单条消息约 **4096 字节** 上限,`send-wecom.ps1` 已做截断
- Webhook / API Key **勿提交 Git**,只用 `.env`
- 企微 markdown 为子集;超长报告**自动分条推送**(默认 4096 bytes/条
- 完整配置项见 [`.env.example`](.env.example)
- 开发测试:`pip install -r requirements-dev.txt` → `pytest`

View File

@@ -5,7 +5,7 @@ WECOM_BOT_SECRET=your-bot-secret
# Cursor SDK@ 机器人后的通用任务由 Cursor 执行)
CURSOR_API_KEY=cursor_...
CURSOR_CWD=d:\LY\test\tech
CURSOR_CWD=.
CURSOR_MODEL=composer-2.5
CURSOR_TIMEOUT=600

143
bot/README.md Normal file
View File

@@ -0,0 +1,143 @@
# bot/ — 企微 API 模式智能机器人
> **EXPERIMENTAL** — 本模块不在 daily-robots v1 开源支持范围内。
> 主产品为根目录 **早报推送**webhook见 [README](../README.md)。
> Bot 需独立 venv、企微 API 凭证、常驻进程;问题请自行排查或提 issue 标注 `bot`。
在企微里 @ 机器人即可:
- **快查**`trending 10``hot 10``搜索 react`(本地 skills 数据,秒回)
- **截图预览**`preview` / `截图`(基于 `CURSOR_CWD` 启动前端并发图)
- **通用任务**:任意自然语言需求,由 **Cursor Agent** 执行并回传结果
数据层与早报共用 [`shared/skills_data.py`](../shared/skills_data.py)。
---
## 1. 创建 API 模式机器人
1. [企业微信管理后台](https://work.weixin.qq.com/) → **安全与管理****管理工具****智能机器人****创建机器人**
2. 选择 **API 模式创建****使用长连接**
3. 记录 **Bot ID****Secret**Secret 只显示一次,请立即保存)
4. 设置可见范围,将机器人 **添加到目标群** 或允许成员单聊
普通成员路径:工作台 → 智能机器人 → 手动创建 → API 模式 → 长连接
---
## 2. 启动本地服务
```powershell
cd path\to\daily-robots\bot
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
playwright install chromium
copy .env.example .env
# 编辑 .envWECOM_BOT_ID / WECOM_BOT_SECRET / CURSOR_API_KEY
python main.py
```
服务需 **常驻运行**(本机、服务器或 Docker。长连接模式下机器人进程须在线才能收消息。
---
## 3. 路由模式ROUTING_MODE
| 模式 | 行为 |
|------|------|
| `hybrid`(默认) | `trending`/`hot`/`搜索`/`详情` 走本地快查;其余 @ 消息交给 Cursor |
| `cursor` | 所有消息都交给 Cursor 执行 |
| `skills` | 仅本地 skills 快查 |
**Cursor 任务示例**(群里发送):
```
@test 总结 trending top10并推荐 3 个适合前端团队的 skill
@test 对比 mattpocock/skills 和 obra/superpowers 各有哪些热门 skill
@test 帮我写一段 npx skills add 的安装说明
```
Cursor 在本机 `CURSOR_CWD` 目录下运行(见 `bot/.env`)。复杂任务可能需要 110 分钟流式消息会显示「Cursor 正在执行任务…」。
---
## 4. 前端截图预览API 模式发图)
项目路径读取 `.env` 中的 **`CURSOR_CWD`**。机器人会:
1.`CURSOR_CWD` 检测 `package.json`,若有 `dev` 脚本则执行 `PREVIEW_DEV_COMMAND`(默认 `npm run dev`
2. 等待 `PREVIEW_PORT`(默认 `5173`)就绪,或用 `PREVIEW_URL` 直接访问
3. Playwright 打开页面并截图
4. 通过 API 模式 **上传图片 + 回复 image 消息** 到群
| 命令 | 说明 |
|------|------|
| `preview` / `截图` / `预览` | 访问 `http://127.0.0.1:5173/` 并截图 |
| `preview /login` | 指定路径 |
| `preview / 3000` | 指定端口 |
| `preview http://127.0.0.1:8080/` | 指定完整 URL |
`.env` 可选配置:
```env
CURSOR_CWD=.
PREVIEW_PORT=5173
PREVIEW_URL=http://127.0.0.1:5173/
PREVIEW_DEV_COMMAND=npm run dev
PREVIEW_STARTUP_TIMEOUT=120
```
---
## 4b. 网页操作Playwright 步骤引擎)
支持三种方式定义操作流程,**无需改 Python 代码**
**1. 自然语言(企微里直接说)**
```
@test 访问登录页,输入账号密码,点击登录后进入主页,点击智能体管理菜单然后截图
```
账号密码从 `.env` 读取(`{{PREVIEW_LOGIN_USER}}` / `{{PREVIEW_LOGIN_PASSWORD}}`),勿在群里发密码。
**2. 场景文件 YAML** — 见 `scenarios/`,触发:`@test browser <场景名>`
**3. 消息内 DSL** — 以 `browser:` 开头的多行步骤
支持的步骤:`goto` · `fill` · `click` · `wait` · `screenshot` · `press`
---
## 5. 快查命令
| 命令 | 说明 |
|------|------|
| `trending 10` / `趋势 10` | 近期增长榜 Top N默认 10最大 30 |
| `hot 10` / `实时 10` | 实时热度榜 |
| `all 10` / `总榜 10` | 历史总安装榜 |
| `搜索 react` / `search tdd` | 关键词搜索 |
| `详情 find-skills` | 单个 skill 详情 + 安装命令 |
| `preview` / `截图` | 启动 CURSOR_CWD 前端并截图发群 |
| `帮助` | 命令列表 |
---
## 6. 本地测试(无需企微凭证)
```powershell
cd path\to\daily-robots\bot
python -c "from skills_service import handle_command; print(handle_command('trending 5'))"
```
---
## 配置
| 文件 | 说明 |
|------|------|
| `bot/.env` | `WECOM_BOT_ID``WECOM_BOT_SECRET``CURSOR_API_KEY` 等 |
| `.env.example` | 模板 |
与根目录 `.env`(早报 webhook**相互独立**,勿混用。

View File

@@ -3,11 +3,16 @@
from __future__ import annotations
import os
import sys
from pathlib import Path
from dotenv import load_dotenv
_BOT_DIR = Path(__file__).resolve().parent
_REPO_ROOT = _BOT_DIR.parent
if str(_REPO_ROOT) not in sys.path:
sys.path.insert(0, str(_REPO_ROOT))
load_dotenv(_BOT_DIR / ".env")
load_dotenv(_BOT_DIR / ".env.local", override=True)

View File

@@ -1,32 +1,20 @@
"""skills.sh 数据查询与命令解析"""
"""skills.sh 快查命令解析与企微回复格式化"""
from __future__ import annotations
import json
import logging
import re
import time
import sys
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Literal
import certifi
import httpx
_REPO_ROOT = Path(__file__).resolve().parent.parent
if str(_REPO_ROOT) not in sys.path:
sys.path.insert(0, str(_REPO_ROOT))
logger = logging.getLogger(__name__)
from shared.skills_data import Board, board_items, format_installs, load_feed, warm_feed_cache
FEED_URLS = [
# jsDelivr 在国内通常比 raw.githubusercontent.com 更稳定
"https://cdn.jsdelivr.net/gh/NeverSight/skills.sh_feed@main/data/feed.json",
"https://raw.githubusercontent.com/NeverSight/skills.sh_feed/main/data/feed.json",
]
CACHE_TTL_SECONDS = 600
CACHE_DIR = Path(__file__).resolve().parent / ".cache"
CACHE_FILE = CACHE_DIR / "feed.json"
_cache: dict[str, Any] = {"data": None, "fetched_at": 0.0}
Board = Literal["trending", "hot", "all"]
__all__ = ["Command", "handle_command", "parse_command", "warm_feed_cache"]
@dataclass
@@ -37,72 +25,6 @@ class Command:
query: str = ""
def _fetch_json(url: str) -> dict[str, Any]:
headers = {
"User-Agent": "skills-hot-bot/1.0",
"Accept": "application/json",
}
with httpx.Client(
timeout=httpx.Timeout(20.0, connect=10.0),
verify=certifi.where(),
follow_redirects=True,
) as client:
resp = client.get(url, headers=headers)
resp.raise_for_status()
return resp.json()
def _load_disk_cache() -> dict[str, Any] | None:
if not CACHE_FILE.exists():
return None
try:
return json.loads(CACHE_FILE.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
logger.warning("读取本地缓存失败: %s", exc)
return None
def _save_disk_cache(data: dict[str, Any]) -> None:
CACHE_DIR.mkdir(parents=True, exist_ok=True)
CACHE_FILE.write_text(json.dumps(data, ensure_ascii=False), encoding="utf-8")
def load_feed(force: bool = False) -> dict[str, Any]:
now = time.time()
if not force and _cache["data"] and now - _cache["fetched_at"] < CACHE_TTL_SECONDS:
return _cache["data"]
errors: list[str] = []
for url in FEED_URLS:
for attempt in range(3):
try:
data = _fetch_json(url)
_cache["data"] = data
_cache["fetched_at"] = now
_save_disk_cache(data)
logger.info("skills 数据已更新: %s", url)
return data
except Exception as exc:
msg = f"{url} (#{attempt + 1}): {exc}"
errors.append(msg)
logger.debug("拉取失败 %s", msg)
time.sleep(0.5 * (attempt + 1))
stale = _load_disk_cache()
if stale:
logger.warning("网络不可用,回退到本地缓存")
_cache["data"] = stale
_cache["fetched_at"] = now
return stale
raise RuntimeError(f"无法获取 skills 数据。最近错误: {errors[-1]}")
def warm_feed_cache() -> None:
"""启动时预加载,避免首条消息才触发网络请求。"""
load_feed(force=True)
def _normalize_text(text: str) -> str:
text = re.sub(r"@\S+\s*", "", text)
return text.strip().lower()
@@ -157,19 +79,6 @@ def parse_command(text: str) -> Command:
return Command(kind="search", query=raw, limit=5)
def _format_installs(n: int | float) -> str:
if n >= 1_000_000:
return f"{n / 1_000_000:.1f}M"
if n >= 1_000:
return f"{n / 1_000:.1f}K"
return str(int(n))
def _board_items(feed: dict[str, Any], board: Board) -> list[dict[str, Any]]:
key = {"trending": "topTrending", "hot": "topHot", "all": "topAllTime"}[board]
return feed.get(key, [])
def _board_title(board: Board) -> str:
return {
"trending": "Trending近期增长",
@@ -180,7 +89,7 @@ def _board_title(board: Board) -> str:
def format_list(board: Board, limit: int) -> str:
feed = load_feed()
items = _board_items(feed, board)[:limit]
items = board_items(feed, board)[:limit]
updated = feed.get("updatedAt", "未知")[:10]
lines = [
@@ -192,7 +101,7 @@ def format_list(board: Board, limit: int) -> str:
for i, item in enumerate(items, 1):
title = item.get("title", "?")
source = item.get("source", "?")
installs = _format_installs(item.get("installs", 0))
installs = format_installs(item.get("installs", 0))
desc = item.get("description", "")
if len(desc) > 80:
desc = desc[:77] + "..."
@@ -241,7 +150,7 @@ def format_search(query: str, limit: int) -> str:
for i, item in enumerate(matches, 1):
title = item.get("title", "?")
source = item.get("source", "?")
installs = _format_installs(item.get("installs", 0))
installs = format_installs(item.get("installs", 0))
link = item.get("link", "")
lines.append(f"{i}. **{title}** · {installs} · `{source}`")
if link:
@@ -270,7 +179,7 @@ def format_detail(name: str) -> str:
[
f"**{best.get('title', '?')}**",
f"`{best.get('source', '?')}`",
f"安装量:**{_format_installs(best.get('installs', 0))}**",
f"安装量:**{format_installs(best.get('installs', 0))}**",
"",
desc,
"",

166
config/feeds.yaml Normal file
View File

@@ -0,0 +1,166 @@
version: 1
intl:
categories:
- id: official
name: 厂商官方
icon: 🏢
feeds:
- name: Anthropic Claude 更新
url: https://docs.anthropic.com/en/release-notes/feed
- name: OpenAI
url: https://openai.com/news/rss.xml
- name: Google AI
url: https://blog.google/technology/ai/rss/
- name: DeepMind
url: https://deepmind.google/blog/rss.xml
- name: Meta Engineering
url: https://engineering.fb.com/feed/
- name: Microsoft Research
url: https://www.microsoft.com/en-us/research/feed/
- name: Microsoft Blog
url: https://blogs.microsoft.com/feed/
- name: Cohere
url: https://cohere.com/blog/rss.xml
- name: Cursor Changelog
url: https://cursor.com/changelog/rss.xml
- id: developer
name: Agent / LLM 开发者
icon: 🛠
feeds:
- name: LangChain
url: https://blog.langchain.dev/rss/
- name: Hugging Face
url: https://huggingface.co/blog/feed.xml
- name: Vercel Changelog
url: https://vercel.com/changelog/rss.xml
- name: GitHub Copilot
url: https://github.blog/changelog/label/copilot/feed/
- id: media
name: 综合科技媒体
icon: 📰
feeds:
- name: The Verge AI
url: https://www.theverge.com/rss/ai-artificial-intelligence/index.xml
- name: TechCrunch AI
url: https://techcrunch.com/category/artificial-intelligence/feed/
- name: Ars Technica AI
url: https://arstechnica.com/ai/feed/
- name: Wired AI
url: https://www.wired.com/feed/tag/ai/latest/rss
- name: MIT Tech Review
url: https://www.technologyreview.com/feed/
- name: VentureBeat AI
url: https://venturebeat.com/category/ai/feed/
- id: newsletter
name: Newsletter 日报
icon: ✉️
feeds:
- name: Ben's Bites
url: https://bensbites.substack.com/feed
- name: The Rundown AI
url: https://therundown.substack.com/feed
- name: Latent Space
url: https://www.latent.space/feed
- name: Simon Willison
url: https://simonwillison.net/atom/everything/
- name: Import AI
url: https://importai.substack.com/feed
- name: Last Week in AI
url: https://lastweekin.ai/feed
- name: The Neuron
url: https://www.theneuron.ai/feed
- id: research
name: 研究 / 论文
icon: 📚
feeds:
- name: arXiv cs.CL
url: https://arxiv.org/rss/cs.CL
- name: arXiv cs.AI
url: https://arxiv.org/rss/cs.AI
- name: arXiv cs.LG
url: https://arxiv.org/rss/cs.LG
- id: trending
name: 热点 / 趋势
icon: 🔥
feeds:
- name: Google News · AI
url: https://news.google.com/rss/search?q=artificial+intelligence+OR+LLM+OR+Claude+OR+GPT&hl=en-US&gl=US&ceid=US:en
- name: Google News · Technology
url: https://news.google.com/rss/headlines/section/topic/TECHNOLOGY?hl=en-US&gl=US&ceid=US:en
- name: Techmeme
url: https://www.techmeme.com/feed.xml
- name: HN · Front Page
url: https://hnrss.org/frontpage
- name: HN · 100+ Points
url: https://hnrss.org/newest?points=100
- name: Dev.to · AI
url: https://dev.to/feed/tag/ai
- name: Lobsters
url: https://lobste.rs/rss
- id: community
name: 社区讨论
icon: 💬
feeds:
- name: HN · AI/LLM/Agent
url: https://hnrss.org/newest?q=AI+OR+LLM+OR+Claude+OR+agent+OR+GPT+OR+Gemini
- name: Reddit · LLM/Claude/ML
url: https://old.reddit.com/r/LocalLLaMA+ClaudeAI+MachineLearning+OpenAI/.rss?limit=25
slow: true
cn:
title_keywords:
- 人工智能
- 大模型
- 智能体
- 多模态
- AIGC
- LLM
- GPT
- Claude
- Gemini
- ChatGPT
- OpenAI
- Anthropic
- Copilot
- Agent
- 'AI '
- ' AI'
- AI·
- AI业务
- AI模型
- AI助手
- AI工具
- AI编程
- AI 编程
- AI版
- AI Agent
- 推理模型
- 深度学习
- 机器学习
- Function Calling
categories:
- id: media
name: AI 专业媒体
icon: 📰
feeds:
- name: 量子位
url: https://www.qbitai.com/feed
- name: InfoQ 中文
url: https://www.infoq.cn/feed/AI
- id: tech
name: 综合科技
icon: 📱
feeds:
- name: 36氪
url: https://36kr.com/feed
ai_filter: true
- name: 雷锋网
url: https://www.leiphone.com/feed
- name: Google News · AI
url: https://news.google.com/rss/search?q=人工智能+OR+大模型+OR+Agent+OR+LLM&hl=zh-CN&gl=CN&ceid=CN:zh-Hans
- id: dev
name: 开发者社区
icon: 💻
feeds:
- name: 掘金
url: https://juejin.cn/rss
ai_filter: true

View File

@@ -0,0 +1,10 @@
# 标题或摘要命中任一词则丢弃(不区分大小写)
words:
- 赌博
- 六合彩
- 网赌
- 色情
- 裸聊
- 代孕
- 办证
- 刷单兼职

View File

@@ -125,10 +125,20 @@ def run_agent_workflow(
trends = analyze_trends(llm_input, date_str=date_str)
if not trends:
return None
return write_wecom_report(
md = write_wecom_report(
llm_input,
trends,
date_str=date_str,
time_str=time_str,
updated=updated,
)
if md is None:
logger.warning("Agent Step2 首次失败,重试一次")
md = write_wecom_report(
llm_input,
trends,
date_str=date_str,
time_str=time_str,
updated=updated,
)
return md

View File

@@ -3,13 +3,11 @@
from __future__ import annotations
import os
import sys
from pathlib import Path
from dotenv import load_dotenv
ROOT = Path(__file__).resolve().parent.parent
BOT_DIR = ROOT / "bot"
OUTPUT_DIR = ROOT / "output"
LOG_DIR = ROOT / "logs"
CACHE_DIR = ROOT / ".cache"
@@ -49,12 +47,6 @@ load_dotenv(ROOT / ".env")
load_dotenv(ROOT / ".env.local", override=True)
def ensure_bot_on_path() -> None:
bot = str(BOT_DIR)
if bot not in sys.path:
sys.path.insert(0, bot)
def _clean_env_value(raw: str | None) -> str | None:
if raw is None:
return None

142
daily/cursor_bridge.py Normal file
View File

@@ -0,0 +1,142 @@
"""Windows 兼容的 Cursor SDK bridgedaily 包自用,不依赖 bot/)。"""
from __future__ import annotations
import codecs
import json
import logging
import os
import subprocess
import threading
import time
from pathlib import Path
from typing import Any, Mapping
from daily.config import ROOT, env
logger = logging.getLogger(__name__)
READY_LINE_PREFIX = "cursor-sdk-bridge ready "
_bridge_lock = threading.Lock()
_bridge_process: subprocess.Popen[bytes] | None = None
def cursor_cwd() -> str:
return env("DAILY_CURSOR_CWD") or env("CURSOR_CWD") or str(ROOT)
def _parse_discovery_line(line: str) -> Mapping[str, Any] | None:
if not line.startswith(READY_LINE_PREFIX):
return None
payload = line[len(READY_LINE_PREFIX) :].strip()
loaded = json.loads(payload)
if not isinstance(loaded, dict):
raise RuntimeError("Bridge discovery payload must be an object")
return loaded
def _read_discovery_polling(process: subprocess.Popen[bytes], timeout: float = 60) -> Mapping[str, Any]:
if process.stderr is None:
raise RuntimeError("Bridge stderr unavailable")
fd = process.stderr.fileno()
was_blocking = os.get_blocking(fd)
os.set_blocking(fd, False)
decoder = codecs.getincrementaldecoder("utf-8")(errors="replace")
pending = ""
stderr_lines: list[str] = []
deadline = time.monotonic() + timeout
try:
while time.monotonic() < deadline:
try:
chunk = os.read(fd, 8192)
except BlockingIOError:
chunk = b""
if chunk:
pending += decoder.decode(chunk)
while "\n" in pending:
line, pending = pending.split("\n", 1)
stderr_lines.append(line)
discovery = _parse_discovery_line(line)
if discovery is not None:
return discovery
else:
code = process.poll()
if code is not None:
pending += decoder.decode(b"", final=True)
if pending.strip():
stderr_lines.append(pending.strip())
joined = "\n".join(stderr_lines)[-2000:]
raise RuntimeError(
f"Bridge 启动失败 exit={code}: {joined or '无 stderr 输出'}"
)
time.sleep(0.05)
finally:
os.set_blocking(fd, was_blocking)
raise RuntimeError("等待 Cursor bridge 就绪超时")
def _auth_token_from_discovery(discovery: Mapping[str, Any]) -> str:
token = str(discovery.get("authToken") or "").strip()
if token:
return token
token_file = discovery.get("authTokenFile")
if token_file:
return Path(str(token_file)).read_text(encoding="utf-8").strip()
raise RuntimeError("Bridge discovery 缺少 auth token")
def warm_cursor_bridge(force: bool = False) -> None:
"""启动 cursor-sdk-bridge 并写入 CURSOR_SDK_BRIDGE_* 环境变量。"""
global _bridge_process
with _bridge_lock:
if (
not force
and _bridge_process is not None
and _bridge_process.poll() is None
and os.environ.get("CURSOR_SDK_BRIDGE_URL")
and os.environ.get("CURSOR_SDK_BRIDGE_TOKEN")
):
return
if _bridge_process is not None and _bridge_process.poll() is None:
_bridge_process.terminate()
try:
_bridge_process.wait(timeout=5)
except subprocess.TimeoutExpired:
_bridge_process.kill()
from cursor_sdk._vendor import resolve_bridge_path
cwd = cursor_cwd()
os.environ["CURSOR_CWD"] = cwd
argv = [resolve_bridge_path(), "--workspace", cwd]
logger.info("启动 Cursor bridge workspace=%s", cwd)
process = subprocess.Popen(
argv,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
)
try:
discovery = _read_discovery_polling(process)
except Exception:
process.kill()
process.wait(timeout=5)
raise
url = str(discovery.get("url") or "").strip()
if not url:
host = str(discovery.get("host") or "127.0.0.1")
port = discovery.get("port")
url = f"http://{host}:{port}"
token = _auth_token_from_discovery(discovery)
os.environ["CURSOR_SDK_BRIDGE_URL"] = url
os.environ["CURSOR_SDK_BRIDGE_TOKEN"] = token
_bridge_process = process
logger.info("Cursor bridge 就绪: %s", url)

68
daily/cursor_client.py Normal file
View File

@@ -0,0 +1,68 @@
"""Cursor SDK 调用daily 包专用)。"""
from __future__ import annotations
import os
from cursor_sdk import Agent, AgentOptions, Client, CursorAgentError, LocalAgentOptions
from daily.config import env, env_int
from daily.cursor_bridge import cursor_cwd, warm_cursor_bridge
_sdk_client: Client | None = None
_sdk_client_key: tuple[str, str, float] | None = None
def cursor_timeout_seconds() -> float:
"""Cursor bridge unary/stream 超时(秒);与 bot 共用 CURSOR_TIMEOUT。"""
return float(env_int("CURSOR_TIMEOUT", 600))
def _cursor_sdk_client() -> Client:
"""带自定义超时的 bridge ClientSDK 默认 unary 仅 60s复杂 Agent 任务易超时)。"""
global _sdk_client, _sdk_client_key
warm_cursor_bridge()
url = os.environ.get("CURSOR_SDK_BRIDGE_URL", "")
token = os.environ.get("CURSOR_SDK_BRIDGE_TOKEN", "")
timeout = cursor_timeout_seconds()
key = (url, token, timeout)
if _sdk_client is not None and _sdk_client_key == key:
return _sdk_client
if _sdk_client is not None:
_sdk_client.close()
_sdk_client = Client(
base_url=url,
auth_token=token,
unary_timeout=timeout,
stream_timeout=timeout,
allow_api_key_env_fallback=False,
)
_sdk_client_key = key
return _sdk_client
def cursor_chat(system: str, user: str) -> str:
api_key = (env("CURSOR_API_KEY") or "").strip()
if not api_key:
return ""
client = _cursor_sdk_client()
cwd = cursor_cwd()
model = env("CURSOR_MODEL") or "composer-2.5"
prompt = f"{system}\n\n{user}"
try:
result = Agent.prompt(
prompt,
AgentOptions(
api_key=api_key,
model=model,
local=LocalAgentOptions(cwd=cwd),
),
client=client,
)
except CursorAgentError as exc:
raise RuntimeError(f"LLM 调用失败:{exc.message}") from exc
if result.status == "error":
raise RuntimeError(f"LLM 调用失败:{result.result or '未知错误'}")
return (result.result or "").strip()

View File

@@ -2,475 +2,58 @@
from __future__ import annotations
import json
import re
import logging
import sys
import xml.etree.ElementTree as ET
from collections import defaultdict
from datetime import datetime, timezone, timedelta
from pathlib import Path
from typing import Any
import certifi
import httpx
import logging
logger = logging.getLogger(__name__)
from daily.config import (
CACHE_DIR,
LOG_DIR,
OUTPUT_DIR,
SNAPSHOT_FILE,
ensure_bot_on_path,
env,
env_int,
full_desc_limit,
news_summary_limit,
wecom_skill_desc_limit,
)
from daily.format_wecom import build_wecom_report, finalize_wecom_skill_groups, replace_wecom_skill_sections
from daily.agent_workflow import is_agent_mode, run_agent_workflow
from daily.delta import compare_depth
from daily.config import LOG_DIR, OUTPUT_DIR, env
from daily.cursor_editor import (
apply_descriptions,
is_enabled as cursor_editor_enabled,
run_editorial,
theme_line_from_editorial,
)
from daily.github.auth import github_html_headers
from daily.github.search import fetch_emerging_repos, fetch_topic_hot_repos
from daily.github.trending import fetch_github_trending, trending_data_source_note
from daily.localize import LocalizeJob, localize_descriptions, needs_chinese
from daily.format_wecom import build_wecom_report, finalize_wecom_skill_groups, replace_wecom_skill_sections
from daily.github.trending import trending_data_source_note
from daily.news.fetch import (
fetch_ai_news,
fetch_cn_ai_news,
format_cn_news_section,
format_news_section,
prepare_wecom_cn_news_items,
prepare_wecom_news_items,
)
from daily.report_data import (
build_full_payload,
build_llm_input,
data_json_path,
save_json,
from daily.pipeline.collect import collect_report_context
from daily.pipeline.formatters import (
build_highlights,
fetch_latest_release_title,
format_github_repo_section,
format_skill_section,
prepare_github_item,
prepare_skill_item,
)
from daily.skills_board import load_boards
from daily.pipeline.localize import localize_descriptions_in_place
from daily.pipeline.snapshot import load_snapshot, save_snapshot
from daily.pipeline.themes import detect_theme_line, theme_clusters
from daily.report_data import build_full_payload, data_json_path, save_json
from daily.skills_group import group_skills_by_source
ensure_bot_on_path()
from skills_service import _format_installs, load_feed # noqa: E402
THEME_RULES: list[tuple[str, str, list[str]]] = [
("🎬", "AI 多媒体 / 视频", ["runcomfy", "remotion", "video", "seedance", "inpaint", "lipsync"]),
("🔧", "工程协作 / Skill 元能力", ["grill", "tdd", "architecture", "find-skills", "to-issues"]),
("📱", "飞书 / Lark", ["lark", "feishu"]),
("📣", "内容营销", ["viral", "tiktok", "instagram", "reels"]),
("🎨", "设计 / 前端", ["frontend", "design", "ui-ux", "tailwind"]),
]
logger = logging.getLogger(__name__)
def _now_cst() -> datetime:
return datetime.now(timezone(timedelta(hours=8)))
def _short_desc(text: str, limit: int = 72) -> str:
text = re.sub(r"\s+", " ", text or "").strip()
if limit <= 0 or len(text) <= limit:
return text
return text[: limit - 3] + "..."
def _archive_desc(text: str) -> str:
return _short_desc(text, full_desc_limit())
def _wecom_desc(text: str, limit: int = 36) -> str:
return _short_desc(text, limit)
def _localize_descriptions_in_place(
trending: list[dict[str, Any]],
hot: list[dict[str, Any]],
github_trending: list[dict[str, Any]],
github_emerging: list[dict[str, Any]],
github_topic: list[dict[str, Any]],
ai_news: dict[str, Any],
cn_ai_news: dict[str, Any] | None = None,
) -> None:
full_limit = full_desc_limit()
news_limit = news_summary_limit()
jobs: list[LocalizeJob] = []
seen_skill: set[str] = set()
for item in trending + hot:
sid = _skill_id(item)
if sid in seen_skill:
continue
seen_skill.add(sid)
desc = (item.get("description") or "").strip()
if desc:
jobs.append(LocalizeJob(f"skill:{sid}", desc, full_limit))
seen_repo: set[str] = set()
for repo_list in (github_trending, github_emerging, github_topic):
for item in repo_list:
repo = item.get("repo", "")
if not repo or repo in seen_repo:
continue
seen_repo.add(repo)
desc = (item.get("description") or "").strip()
if desc:
jobs.append(LocalizeJob(f"github:{repo}", desc, full_limit))
if ai_news.get("enabled"):
seen_news: set[str] = set()
for item in ai_news.get("flat") or []:
link = item.get("link", "")
if not link or link in seen_news:
continue
seen_news.add(link)
summary = (item.get("summary") or "").strip()
if summary:
jobs.append(LocalizeJob(f"news:{link}", summary, news_limit))
zh_map = localize_descriptions(jobs, archive=True)
if not zh_map and not jobs:
return
def _apply_zh(mapping: dict[str, str]) -> None:
for item in trending + hot:
key = f"skill:{_skill_id(item)}"
if key in mapping:
item["description"] = mapping[key]
for repo_list in (github_trending, github_emerging, github_topic):
for item in repo_list:
key = f"github:{item.get('repo', '')}"
if key in mapping:
item["description"] = mapping[key]
if ai_news.get("enabled"):
for cat in ai_news.get("categories") or []:
for item in cat.get("items") or []:
key = f"news:{item.get('link', '')}"
if key in mapping:
item["summary"] = mapping[key]
for item in ai_news.get("flat") or []:
key = f"news:{item.get('link', '')}"
if key in mapping:
item["summary"] = mapping[key]
_apply_zh(zh_map)
# 仍为英文的条目再译一轮(长描述或批次失败时)
retry_jobs: list[LocalizeJob] = []
seen_skill.clear()
for item in trending + hot:
sid = _skill_id(item)
if sid in seen_skill:
continue
seen_skill.add(sid)
desc = (item.get("description") or "").strip()
if needs_chinese(desc):
retry_jobs.append(LocalizeJob(f"skill:{sid}", desc, full_limit))
seen_repo.clear()
for repo_list in (github_trending, github_emerging, github_topic):
for item in repo_list:
repo = item.get("repo", "")
if not repo or repo in seen_repo:
continue
seen_repo.add(repo)
desc = (item.get("description") or "").strip()
if needs_chinese(desc):
retry_jobs.append(LocalizeJob(f"github:{repo}", desc, full_limit))
if ai_news.get("enabled"):
seen_news.clear()
for item in ai_news.get("flat") or []:
link = item.get("link", "")
if not link or link in seen_news:
continue
seen_news.add(link)
summary = (item.get("summary") or "").strip()
if needs_chinese(summary):
retry_jobs.append(LocalizeJob(f"news:{link}", summary, news_limit))
if retry_jobs:
_apply_zh(localize_descriptions(retry_jobs, archive=True))
def _skill_id(item: dict[str, Any]) -> str:
return str(item.get("id") or f"{item.get('source')}/{item.get('title')}")
def _load_snapshot() -> set[str]:
if not SNAPSHOT_FILE.exists():
return set()
try:
data = json.loads(SNAPSHOT_FILE.read_text(encoding="utf-8"))
return set(str(x) for x in (data.get("skill_ids") or []))
except (OSError, json.JSONDecodeError):
return set()
def _save_snapshot(feed: dict[str, Any], date_str: str) -> None:
CACHE_DIR.mkdir(parents=True, exist_ok=True)
ids: list[str] = []
for board in ("topTrending", "topHot"):
for item in feed.get(board, [])[:20]:
sid = _skill_id(item)
if sid not in ids:
ids.append(sid)
SNAPSHOT_FILE.write_text(
json.dumps({"date": date_str, "skill_ids": ids}, ensure_ascii=False, indent=2),
encoding="utf-8",
)
def _prepare_skill_item(item: dict[str, Any], prev_ids: set[str], rank: int) -> dict[str, Any]:
badge = ""
sid = _skill_id(item)
if sid not in prev_ids and prev_ids:
badge = "🆕"
elif rank == 1:
badge = "👑"
installs_fmt = item.get("installs_fmt") or _format_installs(item.get("installs", 0))
title = item.get("source", "?") if item.get("cluster") else item.get("title", "?")
desc = item.get("wecom_desc") or item.get("description") or item.get("cluster_titles") or ""
limit = wecom_skill_desc_limit()
desc_short = desc if item.get("wecom_desc") or limit <= 0 else _wecom_desc(desc, limit)
return {
"title": title,
"source": item.get("source", "?"),
"installs_fmt": installs_fmt,
"link": item.get("link", ""),
"desc_short": desc_short,
"badge": badge,
"cluster": bool(item.get("cluster")),
"cluster_count": item.get("cluster_count"),
"cluster_titles": item.get("cluster_titles"),
}
def _detect_theme_line(feed: dict[str, Any]) -> str:
scores: dict[str, int] = defaultdict(int)
for board in ("topTrending", "topHot"):
for rank, item in enumerate(feed.get(board, [])[:10], 1):
haystack = " ".join(
[item.get("title", ""), item.get("source", ""), item.get("description", "")]
).lower()
for _icon, label, keywords in THEME_RULES:
if any(k in haystack for k in keywords):
scores[label] += max(1, 11 - rank)
break
if not scores:
return "**今日主题**Agent Skills 生态持续活跃"
return f"**今日主题**{max(scores.items(), key=lambda x: x[1])[0]}"
def _build_highlights(
trending: list[dict[str, Any]],
hot: list[dict[str, Any]],
github_trending: list[dict[str, Any]],
github_emerging: list[dict[str, Any]],
ai_news: dict[str, Any] | None = None,
cn_ai_news: dict[str, Any] | None = None,
) -> list[str]:
points: list[str] = []
if ai_news and ai_news.get("enabled"):
top_news = prepare_wecom_news_items(ai_news)
if top_news:
n0 = top_news[0]
pub = f" · {n0['published_fmt']}" if n0.get("published_fmt") else ""
points.append(
f"🌍 AI 时讯 [{n0['title']}]({n0['link']})`{n0.get('source_name', '?')}`{pub}"
)
elif ai_news.get("flat"):
n0 = ai_news["flat"][0]
pub = f" · {n0['published_fmt']}" if n0.get("published_fmt") else ""
points.append(f"🌍 AI 时讯 [{n0['title']}]({n0['link']})`{n0.get('source_name', '?')}`{pub}")
if cn_ai_news and cn_ai_news.get("enabled"):
top_cn = prepare_wecom_cn_news_items(cn_ai_news)
if top_cn:
n0 = top_cn[0]
pub = f" · {n0['published_fmt']}" if n0.get("published_fmt") else ""
points.append(
f"🇨🇳 国内 AI [{n0['title']}]({n0['link']})`{n0.get('source_name', '?')}`{pub}"
)
elif cn_ai_news.get("flat"):
n0 = cn_ai_news["flat"][0]
pub = f" · {n0['published_fmt']}" if n0.get("published_fmt") else ""
points.append(
f"🇨🇳 国内 AI [{n0['title']}]({n0['link']})`{n0.get('source_name', '?')}`{pub}"
)
if trending:
t0 = trending[0]
points.append(f"📈 Skills 榜首 **{t0.get('title')}**{_format_installs(t0.get('installs', 0))}")
if github_trending:
g0 = github_trending[0]
stars = g0.get("stars_today_fmt", "")
total = g0.get("total_stars_fmt", "")
star_hint = f"+{stars} today · " if stars else (f"{total} · " if total else "")
points.append(f"🐙 GitHub Trending [{g0['repo']}]({g0['url']}){star_hint}{g0.get('language', '')}")
if github_emerging:
e0 = github_emerging[0]
points.append(f"🌱 新兴 [{e0['repo']}]({e0['url']})(⭐ {e0.get('total_stars_fmt', '?')}")
elif hot:
h0 = hot[0]
points.append(f"🔥 Skills Hot 榜首 **{h0.get('title')}**1H {_format_installs(h0.get('installs', 0))}")
while len(points) < 3 and len(trending) > len(points):
item = trending[len(points)]
points.append(f"✨ **{item.get('title')}** · `{item.get('source')}`")
return points[:3]
def _prepare_github_item(item: dict[str, Any]) -> dict[str, Any]:
return {**item, "desc_short": _wecom_desc(item.get("description", ""), 40)}
def _fetch_latest_release_title(repo: str) -> str | None:
atom_url = f"https://github.com/{repo}/releases.atom"
try:
with httpx.Client(
timeout=12.0,
verify=certifi.where(),
follow_redirects=True,
headers=github_html_headers(),
) as client:
resp = client.get(atom_url)
if resp.status_code != 200:
return None
root = ET.fromstring(resp.text)
ns = {"a": "http://www.w3.org/2005/Atom"}
entry = root.find("a:entry", ns)
if entry is None:
return None
title = entry.find("a:title", ns)
return title.text.strip() if title is not None and title.text else None
except Exception:
return None
def _theme_clusters(feed: dict[str, Any], limit: int = 5) -> list[tuple[str, list[str]]]:
buckets: dict[str, list[str]] = defaultdict(list)
seen: set[str] = set()
for board in ("topTrending", "topHot"):
for item in feed.get(board, [])[:20]:
item_id = _skill_id(item)
if item_id in seen:
continue
seen.add(item_id)
haystack = " ".join(
[item.get("title", ""), item.get("source", ""), item.get("description", "")]
).lower()
for _icon, theme, keywords in THEME_RULES:
if any(k in haystack for k in keywords):
label = f"**{item.get('title')}** (`{item.get('source')}`)"
if label not in buckets[theme]:
buckets[theme].append(label)
break
return [(theme, examples[:limit]) for theme, examples in buckets.items() if examples]
def _format_github_repo_section(repos: list[dict[str, Any]], *, show_created: bool = False) -> list[str]:
lines: list[str] = []
for i, repo in enumerate(repos, 1):
lang = repo.get("language") or ""
stars_today = repo.get("stars_today_fmt") or ""
total = repo.get("total_stars_fmt") or ""
created = repo.get("created_at") or ""
meta_parts = [lang]
if stars_today:
meta_parts.append(f"+{stars_today} today")
if total:
meta_parts.append(f"总 ⭐ {total}")
if show_created and created:
meta_parts.append(f"创建于 {created}")
lines.append(f"{i}. **[{repo['repo']}]({repo['url']})** · {' · '.join(meta_parts)}")
desc = _archive_desc(repo.get("description", ""))
if desc:
lines.append(f" - {desc}")
lines.append("")
return lines
def _format_skill_section(items: list[dict[str, Any]], *, hot: bool = False) -> list[str]:
lines: list[str] = []
for i, item in enumerate(items, 1):
skill_id = item.get("id") or f"{item.get('source', '?')}/{item.get('title', '?')}"
link = item.get("link", "")
installs = _format_installs(item.get("installs", 0))
meta = f"1H {installs}" if hot else f"总安装 {installs}"
if link:
lines.append(f"{i}. **[{skill_id}]({link})** · {meta}")
else:
lines.append(f"{i}. **{skill_id}** · {meta}")
desc = _archive_desc(item.get("description", ""))
if desc:
lines.append(f" - {desc}")
lines.append("")
return lines
def generate_report() -> tuple[str, str, Path, Path]:
trending_n = env_int("DAILY_TRENDING_LIMIT", 150)
hot_n = max(env_int("DAILY_HOT_LIMIT", 150), compare_depth())
compare_n = compare_depth()
skill_pool = max(10, env_int("DAILY_WECOM_SKILL_POOL", 200))
wecom_trending = env_int("DAILY_WECOM_TRENDING", 10)
wecom_hot = env_int("DAILY_WECOM_HOT", 10)
github_limit = env_int("DAILY_GITHUB_TRENDING_LIMIT", 10)
wecom_github = env_int("DAILY_WECOM_GITHUB_TRENDING", env_int("DAILY_WECOM_REPOS", 10))
github_fetch_n = max(github_limit, compare_n, wecom_github)
emerging_limit = env_int("DAILY_GITHUB_EMERGING_LIMIT", 10)
wecom_emerging = env_int("DAILY_WECOM_GITHUB_EMERGING", 10)
emerging_fetch_n = max(emerging_limit, compare_n, wecom_emerging)
topic_limit = env_int("DAILY_GITHUB_TOPIC_LIMIT", 10)
wecom_topic = env_int("DAILY_WECOM_GITHUB_TOPIC", 10)
topic_fetch_n = max(topic_limit, compare_n, wecom_topic)
feed = load_feed(force=True)
prev_ids = _load_snapshot()
ctx = collect_report_context(_now_cst())
now = _now_cst()
date_str = now.strftime("%Y-%m-%d")
time_str = now.strftime("%H:%M") + " (UTC+8)"
updated = (feed.get("updatedAt") or "")[:10]
prev_ids = load_snapshot()
trending, hot = load_boards(feed, trending_limit=trending_n, hot_limit=hot_n)
github_trending = fetch_github_trending(github_fetch_n)
seen_repos = {r["repo"] for r in github_trending}
github_emerging = fetch_emerging_repos(emerging_fetch_n, exclude=seen_repos)
seen_repos.update(r["repo"] for r in github_emerging)
topic_name, github_topic = fetch_topic_hot_repos(topic_fetch_n, exclude=seen_repos)
ai_news = fetch_ai_news()
cn_ai_news = fetch_cn_ai_news()
wecom_limits = {
"trending": wecom_trending,
"hot": wecom_hot,
"trending_pool": skill_pool,
"hot_pool": skill_pool,
"github": wecom_github,
"emerging": wecom_emerging,
"topic": wecom_topic,
"ai_news": env_int("DAILY_WECOM_AI_NEWS", 10),
"cn_ai_news": env_int("DAILY_WECOM_CN_AI_NEWS", 8),
}
llm_input = build_llm_input(
date_str=date_str,
updated=updated,
trending=trending,
hot=hot,
github_trending=github_trending,
github_emerging=github_emerging,
github_topic=github_topic,
topic_name=topic_name,
ai_news=ai_news,
cn_ai_news=cn_ai_news,
wecom_limits=wecom_limits,
)
save_json(
data_json_path(date_str),
data_json_path(ctx.date_str),
build_full_payload(
llm_input,
ctx.llm_input,
meta={
"generated_at": now.isoformat(),
"report_mode": "agent" if is_agent_mode() else "classic",
@@ -482,10 +65,10 @@ def generate_report() -> tuple[str, str, Path, Path]:
agent_wecom: str | None = None
if is_agent_mode():
agent_wecom = run_agent_workflow(
llm_input,
date_str=date_str,
time_str=time_str,
updated=updated,
ctx.llm_input,
date_str=ctx.date_str,
time_str=ctx.time_str,
updated=ctx.updated,
)
if not agent_wecom:
logger.warning("Agent 工作流失败,回退 classic 模式")
@@ -493,79 +76,88 @@ def generate_report() -> tuple[str, str, Path, Path]:
editorial_theme: str | None = None
editorial_highlights: list[str] | None = None
if agent_wecom is None:
editorial = run_editorial(llm_input, date_str=date_str)
editorial = run_editorial(ctx.llm_input, date_str=ctx.date_str)
if editorial:
apply_descriptions(
trending=trending,
hot=hot,
github_trending=github_trending,
github_emerging=github_emerging,
github_topic=github_topic,
ai_news=ai_news,
cn_ai_news=cn_ai_news,
trending=ctx.trending,
hot=ctx.hot,
github_trending=ctx.github_trending,
github_emerging=ctx.github_emerging,
github_topic=ctx.github_topic,
ai_news=ctx.ai_news,
cn_ai_news=ctx.cn_ai_news,
descriptions=editorial.get("descriptions") or {},
)
editorial_theme = theme_line_from_editorial(editorial) or None
hl = editorial.get("highlights") or []
editorial_highlights = hl if hl else None
# 完整版归档:中文化 + 加长摘要(已是中文的条目会跳过翻译)
_localize_descriptions_in_place(
trending, hot, github_trending, github_emerging, github_topic, ai_news, cn_ai_news
localize_descriptions_in_place(
ctx.trending,
ctx.hot,
ctx.github_trending,
ctx.github_emerging,
ctx.github_topic,
ctx.ai_news,
ctx.cn_ai_news,
)
themes = _theme_clusters(feed)
limits = ctx.limits
themes = theme_clusters(ctx.feed)
lines = [
f"# 早报 · {date_str}",
f"# 早报 · {ctx.date_str}",
"",
f"> 生成时间:{now.strftime('%Y-%m-%d %H:%M')} (UTC+8) ",
f"> skills 数据更新:{updated} ",
f"> skills 数据更新:{ctx.updated} ",
"> 数据来源:[skills.sh/trending](https://skills.sh/trending) · [skills.sh/hot](https://skills.sh/hot) · 国际/国内 AI RSS",
"",
"---",
"",
f"## 一、Skills Trending Top {trending_n}",
f"## 一、Skills Trending Top {limits.trending_n}",
"",
*_format_skill_section(trending),
*format_skill_section(ctx.trending),
"---",
"",
f"## 二、Skills Hot Top {hot_n}",
f"## 二、Skills Hot Top {limits.hot_n}",
"",
*_format_skill_section(hot, hot=True),
*format_skill_section(ctx.hot, hot=True),
"",
"---",
"",
f"## 三、GitHub Trending Top {github_limit}",
f"## 三、GitHub Trending Top {limits.github_limit}",
"",
trending_data_source_note(),
"",
]
if github_trending:
lines.extend(_format_github_repo_section(github_trending))
if ctx.github_trending:
lines.extend(format_github_repo_section(ctx.github_trending))
else:
lines.append("*GitHub Trending 获取失败,请检查网络或配置 GITHUB_TOKEN。*")
lines.append("")
lines.extend(["---", "", f"## 四、新兴项目 Top {emerging_limit}", "", "> 数据来源GitHub Search API需 `GITHUB_TOKEN`", ""])
if github_emerging:
lines.extend(_format_github_repo_section(github_emerging, show_created=True))
lines.extend(
["---", "", f"## 四、新兴项目 Top {limits.emerging_limit}", "", "> 数据来源GitHub Search API需 `GITHUB_TOKEN`", ""]
)
if ctx.github_emerging:
lines.extend(format_github_repo_section(ctx.github_emerging, show_created=True))
else:
lines.append("*新兴项目获取失败或未配置 GITHUB_TOKEN。*")
lines.append("")
lines.extend(["---", "", f"## 五、Topic `{topic_name}` Top {topic_limit}", "", "> 数据来源GitHub Search API需 `GITHUB_TOKEN`", ""])
if github_topic:
lines.extend(_format_github_repo_section(github_topic))
lines.extend(
["---", "", f"## 五、Topic `{ctx.topic_name}` Top {limits.topic_limit}", "", "> 数据来源GitHub Search API需 `GITHUB_TOKEN`", ""]
)
if ctx.github_topic:
lines.extend(format_github_repo_section(ctx.github_topic))
else:
lines.append(f"*Topic `{topic_name}` 热点获取失败或未配置 GITHUB_TOKEN。*")
lines.append(f"*Topic `{ctx.topic_name}` 热点获取失败或未配置 GITHUB_TOKEN。*")
lines.append("")
section_no = 6
lines.extend(format_news_section(ai_news, section_no=section_no))
lines.extend(format_news_section(ctx.ai_news, section_no=section_no))
section_no += 1
lines.extend(format_cn_news_section(cn_ai_news, section_no=section_no))
lines.extend(format_cn_news_section(ctx.cn_ai_news, section_no=section_no))
section_no += 1
watch = (env("GITHUB_REPOS") or "").strip()
@@ -573,7 +165,7 @@ def generate_report() -> tuple[str, str, Path, Path]:
lines.extend(["---", "", f"## {section_no}、关注仓库 Release", ""])
section_no += 1
for repo in [r.strip() for r in watch.split(",") if r.strip()]:
release = _fetch_latest_release_title(repo)
release = fetch_latest_release_title(repo)
lines.append(f"- **{repo}**{release or '暂无 release'}")
lines.append("")
@@ -584,8 +176,8 @@ def generate_report() -> tuple[str, str, Path, Path]:
lines.append(f"- {ex}")
lines.append("")
pick_src = trending[0].get("source", "") if trending else ""
pick_name = trending[0].get("title", "") if trending else ""
pick_src = ctx.trending[0].get("source", "") if ctx.trending else ""
pick_name = ctx.trending[0].get("title", "") if ctx.trending else ""
pick_command = (
f"npx skills add {pick_src}/{pick_name}"
if pick_src and pick_name
@@ -593,56 +185,69 @@ def generate_report() -> tuple[str, str, Path, Path]:
)
lines.extend(["---", "", "## 安装示例", "", "```bash"])
for item in trending[:4]:
for item in ctx.trending[:4]:
src, name = item.get("source", ""), item.get("title", "")
if src and name:
lines.append(f"npx skills add {src}/{name}")
lines.extend(["```", "", f"*企微短版见 `output/{date_str}.wecom.md`*"])
lines.extend(["```", "", f"*企微短版见 `output/{ctx.date_str}.wecom.md`*"])
markdown = "\n".join(lines)
if agent_wecom:
gt = group_skills_by_source(trending, limit=wecom_trending, pool_size=skill_pool)
gh = group_skills_by_source(hot, limit=wecom_hot, pool_size=skill_pool)
gt = group_skills_by_source(
ctx.trending, limit=limits.wecom_trending, pool_size=limits.skill_pool
)
gh = group_skills_by_source(ctx.hot, limit=limits.wecom_hot, pool_size=limits.skill_pool)
wecom_md = replace_wecom_skill_sections(agent_wecom, trending=gt, hot=gh)
else:
wecom_md = build_wecom_report(
date_str=date_str,
time_str=time_str,
updated=updated,
date_str=ctx.date_str,
time_str=ctx.time_str,
updated=ctx.updated,
highlights=editorial_highlights
or _build_highlights(trending, hot, github_trending, github_emerging, ai_news, cn_ai_news),
theme_line=editorial_theme or _detect_theme_line(feed),
ai_news=prepare_wecom_news_items(ai_news),
cn_ai_news=prepare_wecom_cn_news_items(cn_ai_news),
or build_highlights(
ctx.trending,
ctx.hot,
ctx.github_trending,
ctx.github_emerging,
ctx.ai_news,
ctx.cn_ai_news,
),
theme_line=editorial_theme or detect_theme_line(ctx.feed),
ai_news=prepare_wecom_news_items(ctx.ai_news),
cn_ai_news=prepare_wecom_cn_news_items(ctx.cn_ai_news),
trending=[
_prepare_skill_item(item, prev_ids, r)
prepare_skill_item(item, prev_ids, r)
for r, item in enumerate(
finalize_wecom_skill_groups(
group_skills_by_source(trending, limit=wecom_trending, pool_size=skill_pool)
group_skills_by_source(
ctx.trending, limit=limits.wecom_trending, pool_size=limits.skill_pool
)
),
1,
)
],
hot=[
_prepare_skill_item(item, prev_ids, r)
prepare_skill_item(item, prev_ids, r)
for r, item in enumerate(
finalize_wecom_skill_groups(
group_skills_by_source(hot, limit=wecom_hot, pool_size=skill_pool)
group_skills_by_source(
ctx.hot, limit=limits.wecom_hot, pool_size=limits.skill_pool
)
),
1,
)
],
repos=[_prepare_github_item(item) for item in github_trending[:wecom_github]],
emerging=[_prepare_github_item(item) for item in github_emerging[:wecom_emerging]],
topic_name=topic_name,
topic_repos=[_prepare_github_item(item) for item in github_topic[:wecom_topic]],
repos=[prepare_github_item(item) for item in ctx.github_trending[: limits.wecom_github]],
emerging=[prepare_github_item(item) for item in ctx.github_emerging[: limits.wecom_emerging]],
topic_name=ctx.topic_name,
topic_repos=[prepare_github_item(item) for item in ctx.github_topic[: limits.wecom_topic]],
pick_command=pick_command,
)
_save_snapshot(feed, date_str)
save_snapshot(ctx.feed, ctx.date_str)
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
out_md = OUTPUT_DIR / f"{date_str}.md"
out_wecom = OUTPUT_DIR / f"{date_str}.wecom.md"
out_md = OUTPUT_DIR / f"{ctx.date_str}.md"
out_wecom = OUTPUT_DIR / f"{ctx.date_str}.wecom.md"
out_md.write_text(markdown, encoding="utf-8")
out_wecom.write_text(wecom_md, encoding="utf-8")
return markdown, wecom_md, out_md, out_wecom

View File

@@ -3,17 +3,14 @@
from __future__ import annotations
import json
import logging
import os
import re
from typing import Any
import certifi
import httpx
from daily.config import ROOT, env, env_int
logger = logging.getLogger(__name__)
from daily.config import env, env_int
from daily.cursor_client import cursor_chat
_JSON_BLOCK = re.compile(r"```(?:json)?\s*([\s\S]*?)```", re.IGNORECASE)
@@ -44,6 +41,47 @@ def extract_json_object(text: str) -> dict[str, Any]:
return {}
def _has_openai_configured() -> bool:
return bool(env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY"))
def _has_cursor_configured() -> bool:
return bool(env("CURSOR_API_KEY"))
def llm_provider() -> str:
raw = (env("DAILY_LLM_PROVIDER") or "auto").strip().lower()
if raw in {"openai", "cursor"}:
return raw
return "auto"
def resolve_llm_backend() -> str:
"""返回 openai | cursor | 空字符串。"""
provider = llm_provider()
has_openai = _has_openai_configured()
has_cursor = _has_cursor_configured()
if provider == "openai":
if has_openai:
return "openai"
return "cursor" if has_cursor else ""
if provider == "cursor":
if has_cursor:
return "cursor"
return "openai" if has_openai else ""
agent_mode = (env("DAILY_REPORT_MODE") or "").strip().lower() == "agent"
if agent_mode and has_cursor:
return "cursor"
if has_openai:
return "openai"
if has_cursor:
return "cursor"
return ""
def _openai_chat(system: str, user: str) -> str:
api_key = (env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY") or "").strip()
if not api_key:
@@ -70,50 +108,14 @@ def _openai_chat(system: str, user: str) -> str:
return str(data["choices"][0]["message"]["content"] or "").strip()
def _cursor_chat(system: str, user: str) -> str:
api_key = (env("CURSOR_API_KEY") or "").strip()
if not api_key:
return ""
from cursor_sdk import Agent, AgentOptions, CursorAgentError, LocalAgentOptions
from daily.config import ensure_bot_on_path
ensure_bot_on_path()
try:
from bridge_manager import warm_cursor_bridge
except ImportError:
warm_cursor_bridge = lambda: None # noqa: E731
cwd = env("DAILY_CURSOR_CWD") or str(ROOT)
# bridge_manager 读 bot env_config 的 CURSOR_CWD早报侧须先对齐工作目录
os.environ["CURSOR_CWD"] = cwd
warm_cursor_bridge()
model = env("CURSOR_MODEL") or "composer-2.5"
prompt = f"{system}\n\n{user}"
try:
result = Agent.prompt(
prompt,
AgentOptions(
api_key=api_key,
model=model,
local=LocalAgentOptions(cwd=cwd),
),
)
except CursorAgentError as exc:
raise RuntimeError(f"LLM 调用失败:{exc.message}") from exc
if result.status == "error":
raise RuntimeError(f"LLM 调用失败:{result.result or '未知错误'}")
return (result.result or "").strip()
def llm_chat(system: str, user: str) -> str:
"""优先 OpenAI 兼容 API否则 Cursor SDK。"""
if env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY"):
backend = resolve_llm_backend()
if backend == "openai":
return _openai_chat(system, user)
if env("CURSOR_API_KEY"):
return _cursor_chat(system, user)
if backend == "cursor":
return cursor_chat(system, user)
return ""
def has_llm_configured() -> bool:
return bool(env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY") or env("CURSOR_API_KEY"))
return bool(resolve_llm_backend())

View File

@@ -0,0 +1,77 @@
"""敏感词内容过滤。"""
from __future__ import annotations
import logging
from functools import lru_cache
from pathlib import Path
from typing import Any
from daily.config import ROOT, env
logger = logging.getLogger(__name__)
_WORDS_FILE = ROOT / "config" / "sensitive_words.yaml"
def content_filter_enabled() -> bool:
raw = (env("DAILY_CONTENT_FILTER") or "0").strip().lower()
return raw in {"1", "true", "yes", "on"}
@lru_cache(maxsize=1)
def load_sensitive_words() -> tuple[str, ...]:
if not _WORDS_FILE.exists():
return ()
try:
import yaml
except ImportError:
logger.warning("未安装 PyYAML无法读取 %s", _WORDS_FILE)
return ()
try:
data = yaml.safe_load(_WORDS_FILE.read_text(encoding="utf-8"))
except OSError as exc:
logger.warning("读取敏感词配置失败: %s", exc)
return ()
except Exception as exc:
logger.warning("解析 sensitive_words.yaml 失败: %s", exc)
return ()
if not isinstance(data, dict):
return ()
words = data.get("words") or data.get("sensitive_words") or []
if not isinstance(words, list):
return ()
cleaned = tuple(str(word).strip() for word in words if str(word).strip())
return cleaned
def matches_sensitive_text(text: str, words: tuple[str, ...]) -> str | None:
haystack = (text or "").lower()
if not haystack:
return None
for word in words:
needle = word.lower()
if needle and needle in haystack:
return word
return None
def filter_news_items(items: list[dict[str, Any]]) -> tuple[list[dict[str, Any]], int]:
if not content_filter_enabled():
return items, 0
words = load_sensitive_words()
if not words:
return items, 0
kept: list[dict[str, Any]] = []
removed = 0
for item in items:
text = f"{item.get('title', '')} {item.get('summary', '')}"
hit = matches_sensitive_text(text, words)
if hit:
removed += 1
continue
kept.append(item)
if removed:
logger.info("内容过滤移除 %d 条(敏感词)", removed)
return kept, removed

View File

@@ -1,125 +1,10 @@
"""国际 AI 时讯 RSS 源定义(按类别分组)。"""
"""国际 AI 时讯 RSS 源(优先 config/feeds.yaml)。"""
from __future__ import annotations
from dataclasses import dataclass
from daily.news.feeds_loader import load_categories
from daily.news.feeds_types import NewsCategory, NewsFeed
NEWS_CATEGORIES: tuple[NewsCategory, ...] = load_categories("intl")
@dataclass(frozen=True)
class NewsFeed:
name: str
url: str
slow: bool = False # 限速源(如 Reddit串行抓取
ai_filter: bool = False # 综合源仅保留标题命中 AI 关键词的条目
@dataclass(frozen=True)
class NewsCategory:
id: str
name: str
icon: str
feeds: tuple[NewsFeed, ...]
NEWS_CATEGORIES: tuple[NewsCategory, ...] = (
NewsCategory(
id="official",
name="厂商官方",
icon="🏢",
feeds=(
NewsFeed("Anthropic Claude 更新", "https://docs.anthropic.com/en/release-notes/feed"),
NewsFeed("OpenAI", "https://openai.com/news/rss.xml"),
NewsFeed("Google AI", "https://blog.google/technology/ai/rss/"),
NewsFeed("DeepMind", "https://deepmind.google/blog/rss.xml"),
NewsFeed("Meta Engineering", "https://engineering.fb.com/feed/"),
NewsFeed("Microsoft Research", "https://www.microsoft.com/en-us/research/feed/"),
NewsFeed("Microsoft Blog", "https://blogs.microsoft.com/feed/"),
NewsFeed("Cohere", "https://cohere.com/blog/rss.xml"),
NewsFeed("Cursor Changelog", "https://cursor.com/changelog/rss.xml"),
),
),
NewsCategory(
id="developer",
name="Agent / LLM 开发者",
icon="🛠",
feeds=(
NewsFeed("LangChain", "https://blog.langchain.dev/rss/"),
NewsFeed("Hugging Face", "https://huggingface.co/blog/feed.xml"),
NewsFeed("Vercel Changelog", "https://vercel.com/changelog/rss.xml"),
NewsFeed("GitHub Copilot", "https://github.blog/changelog/label/copilot/feed/"),
),
),
NewsCategory(
id="media",
name="综合科技媒体",
icon="📰",
feeds=(
NewsFeed("The Verge AI", "https://www.theverge.com/rss/ai-artificial-intelligence/index.xml"),
NewsFeed("TechCrunch AI", "https://techcrunch.com/category/artificial-intelligence/feed/"),
NewsFeed("Ars Technica AI", "https://arstechnica.com/ai/feed/"),
NewsFeed("Wired AI", "https://www.wired.com/feed/tag/ai/latest/rss"),
NewsFeed("MIT Tech Review", "https://www.technologyreview.com/feed/"),
NewsFeed("VentureBeat AI", "https://venturebeat.com/category/ai/feed/"),
),
),
NewsCategory(
id="newsletter",
name="Newsletter 日报",
icon="✉️",
feeds=(
NewsFeed("Ben's Bites", "https://bensbites.substack.com/feed"),
NewsFeed("The Rundown AI", "https://therundown.substack.com/feed"),
NewsFeed("Latent Space", "https://www.latent.space/feed"),
NewsFeed("Simon Willison", "https://simonwillison.net/atom/everything/"),
NewsFeed("Import AI", "https://importai.substack.com/feed"),
NewsFeed("Last Week in AI", "https://lastweekin.ai/feed"),
NewsFeed("The Neuron", "https://www.theneuron.ai/feed"),
),
),
NewsCategory(
id="research",
name="研究 / 论文",
icon="📚",
feeds=(
NewsFeed("arXiv cs.CL", "https://arxiv.org/rss/cs.CL"),
NewsFeed("arXiv cs.AI", "https://arxiv.org/rss/cs.AI"),
NewsFeed("arXiv cs.LG", "https://arxiv.org/rss/cs.LG"),
),
),
NewsCategory(
id="trending",
name="热点 / 趋势",
icon="🔥",
feeds=(
NewsFeed(
"Google News · AI",
"https://news.google.com/rss/search?q=artificial+intelligence+OR+LLM+OR+Claude+OR+GPT&hl=en-US&gl=US&ceid=US:en",
),
NewsFeed(
"Google News · Technology",
"https://news.google.com/rss/headlines/section/topic/TECHNOLOGY?hl=en-US&gl=US&ceid=US:en",
),
NewsFeed("Techmeme", "https://www.techmeme.com/feed.xml"),
NewsFeed("HN · Front Page", "https://hnrss.org/frontpage"),
NewsFeed("HN · 100+ Points", "https://hnrss.org/newest?points=100"),
NewsFeed("Dev.to · AI", "https://dev.to/feed/tag/ai"),
NewsFeed("Lobsters", "https://lobste.rs/rss"),
),
),
NewsCategory(
id="community",
name="社区讨论",
icon="💬",
feeds=(
NewsFeed(
"HN · AI/LLM/Agent",
"https://hnrss.org/newest?q=AI+OR+LLM+OR+Claude+OR+agent+OR+GPT+OR+Gemini",
),
NewsFeed(
"Reddit · LLM/Claude/ML",
"https://old.reddit.com/r/LocalLLaMA+ClaudeAI+MachineLearning+OpenAI/.rss?limit=25",
slow=True,
),
),
),
)
__all__ = ["NewsCategory", "NewsFeed", "NEWS_CATEGORIES"]

View File

@@ -1,71 +1,11 @@
"""国内 AI 时讯 RSS 源定义(按类别分组)。"""
"""国内 AI 时讯 RSS 源(优先 config/feeds.yaml)。"""
from __future__ import annotations
from daily.news.feeds import NewsCategory, NewsFeed
from daily.news.feeds_loader import load_categories, load_cn_title_keywords
from daily.news.feeds_types import NewsCategory, NewsFeed
# 综合源 ai_filter=True 时,仅保留标题命中以下词之一的条目
CN_AI_TITLE_KEYWORDS: tuple[str, ...] = (
"人工智能",
"大模型",
"智能体",
"多模态",
"AIGC",
"LLM",
"GPT",
"Claude",
"Gemini",
"ChatGPT",
"OpenAI",
"Anthropic",
"Copilot",
"Agent",
"AI ",
" AI",
"AI·",
"AI业务",
"AI模型",
"AI助手",
"AI工具",
"AI编程",
"AI 编程",
"AI版",
"AI Agent",
"推理模型",
"深度学习",
"机器学习",
"Function Calling",
)
CN_AI_TITLE_KEYWORDS: tuple[str, ...] = load_cn_title_keywords()
CN_NEWS_CATEGORIES: tuple[NewsCategory, ...] = load_categories("cn")
CN_NEWS_CATEGORIES: tuple[NewsCategory, ...] = (
NewsCategory(
id="media",
name="AI 专业媒体",
icon="📰",
feeds=(
NewsFeed("量子位", "https://www.qbitai.com/feed"),
NewsFeed("InfoQ 中文", "https://www.infoq.cn/feed/AI"),
),
),
NewsCategory(
id="tech",
name="综合科技",
icon="📱",
feeds=(
NewsFeed("36氪", "https://36kr.com/feed", ai_filter=True),
NewsFeed("雷锋网", "https://www.leiphone.com/feed"),
NewsFeed(
"Google News · AI",
"https://news.google.com/rss/search?q=人工智能+OR+大模型+OR+Agent+OR+LLM&hl=zh-CN&gl=CN&ceid=CN:zh-Hans",
),
),
),
NewsCategory(
id="dev",
name="开发者社区",
icon="💻",
feeds=(
NewsFeed("掘金", "https://juejin.cn/rss", ai_filter=True),
),
),
)
__all__ = ["CN_AI_TITLE_KEYWORDS", "CN_NEWS_CATEGORIES", "NewsCategory", "NewsFeed"]

View File

@@ -0,0 +1,11 @@
"""Built-in RSS defaults when config/feeds.yaml is missing or invalid."""
from __future__ import annotations
from daily.news.feeds_types import NewsCategory, NewsFeed
CN_AI_TITLE_KEYWORDS: tuple[str, ...] = ('人工智能', '大模型', '智能体', '多模态', 'AIGC', 'LLM', 'GPT', 'Claude', 'Gemini', 'ChatGPT', 'OpenAI', 'Anthropic', 'Copilot', 'Agent', 'AI ', ' AI', 'AI·', 'AI业务', 'AI模型', 'AI助手', 'AI工具', 'AI编程', 'AI 编程', 'AI版', 'AI Agent', '推理模型', '深度学习', '机器学习', 'Function Calling')
CN_NEWS_CATEGORIES: tuple[NewsCategory, ...] = (NewsCategory(id='media', name='AI 专业媒体', icon='📰', feeds=(NewsFeed(name='量子位', url='https://www.qbitai.com/feed', slow=False, ai_filter=False), NewsFeed(name='InfoQ 中文', url='https://www.infoq.cn/feed/AI', slow=False, ai_filter=False))), NewsCategory(id='tech', name='综合科技', icon='📱', feeds=(NewsFeed(name='36氪', url='https://36kr.com/feed', slow=False, ai_filter=True), NewsFeed(name='雷锋网', url='https://www.leiphone.com/feed', slow=False, ai_filter=False), NewsFeed(name='Google News · AI', url='https://news.google.com/rss/search?q=人工智能+OR+大模型+OR+Agent+OR+LLM&hl=zh-CN&gl=CN&ceid=CN:zh-Hans', slow=False, ai_filter=False))), NewsCategory(id='dev', name='开发者社区', icon='💻', feeds=(NewsFeed(name='掘金', url='https://juejin.cn/rss', slow=False, ai_filter=True),)))
NEWS_CATEGORIES: tuple[NewsCategory, ...] = (NewsCategory(id='official', name='厂商官方', icon='🏢', feeds=(NewsFeed(name='Anthropic Claude 更新', url='https://docs.anthropic.com/en/release-notes/feed', slow=False, ai_filter=False), NewsFeed(name='OpenAI', url='https://openai.com/news/rss.xml', slow=False, ai_filter=False), NewsFeed(name='Google AI', url='https://blog.google/technology/ai/rss/', slow=False, ai_filter=False), NewsFeed(name='DeepMind', url='https://deepmind.google/blog/rss.xml', slow=False, ai_filter=False), NewsFeed(name='Meta Engineering', url='https://engineering.fb.com/feed/', slow=False, ai_filter=False), NewsFeed(name='Microsoft Research', url='https://www.microsoft.com/en-us/research/feed/', slow=False, ai_filter=False), NewsFeed(name='Microsoft Blog', url='https://blogs.microsoft.com/feed/', slow=False, ai_filter=False), NewsFeed(name='Cohere', url='https://cohere.com/blog/rss.xml', slow=False, ai_filter=False), NewsFeed(name='Cursor Changelog', url='https://cursor.com/changelog/rss.xml', slow=False, ai_filter=False))), NewsCategory(id='developer', name='Agent / LLM 开发者', icon='🛠', feeds=(NewsFeed(name='LangChain', url='https://blog.langchain.dev/rss/', slow=False, ai_filter=False), NewsFeed(name='Hugging Face', url='https://huggingface.co/blog/feed.xml', slow=False, ai_filter=False), NewsFeed(name='Vercel Changelog', url='https://vercel.com/changelog/rss.xml', slow=False, ai_filter=False), NewsFeed(name='GitHub Copilot', url='https://github.blog/changelog/label/copilot/feed/', slow=False, ai_filter=False))), NewsCategory(id='media', name='综合科技媒体', icon='📰', feeds=(NewsFeed(name='The Verge AI', url='https://www.theverge.com/rss/ai-artificial-intelligence/index.xml', slow=False, ai_filter=False), NewsFeed(name='TechCrunch AI', url='https://techcrunch.com/category/artificial-intelligence/feed/', slow=False, ai_filter=False), NewsFeed(name='Ars Technica AI', url='https://arstechnica.com/ai/feed/', slow=False, ai_filter=False), NewsFeed(name='Wired AI', url='https://www.wired.com/feed/tag/ai/latest/rss', slow=False, ai_filter=False), NewsFeed(name='MIT Tech Review', url='https://www.technologyreview.com/feed/', slow=False, ai_filter=False), NewsFeed(name='VentureBeat AI', url='https://venturebeat.com/category/ai/feed/', slow=False, ai_filter=False))), NewsCategory(id='newsletter', name='Newsletter 日报', icon='✉️', feeds=(NewsFeed(name="Ben's Bites", url='https://bensbites.substack.com/feed', slow=False, ai_filter=False), NewsFeed(name='The Rundown AI', url='https://therundown.substack.com/feed', slow=False, ai_filter=False), NewsFeed(name='Latent Space', url='https://www.latent.space/feed', slow=False, ai_filter=False), NewsFeed(name='Simon Willison', url='https://simonwillison.net/atom/everything/', slow=False, ai_filter=False), NewsFeed(name='Import AI', url='https://importai.substack.com/feed', slow=False, ai_filter=False), NewsFeed(name='Last Week in AI', url='https://lastweekin.ai/feed', slow=False, ai_filter=False), NewsFeed(name='The Neuron', url='https://www.theneuron.ai/feed', slow=False, ai_filter=False))), NewsCategory(id='research', name='研究 / 论文', icon='📚', feeds=(NewsFeed(name='arXiv cs.CL', url='https://arxiv.org/rss/cs.CL', slow=False, ai_filter=False), NewsFeed(name='arXiv cs.AI', url='https://arxiv.org/rss/cs.AI', slow=False, ai_filter=False), NewsFeed(name='arXiv cs.LG', url='https://arxiv.org/rss/cs.LG', slow=False, ai_filter=False))), NewsCategory(id='trending', name='热点 / 趋势', icon='🔥', feeds=(NewsFeed(name='Google News · AI', url='https://news.google.com/rss/search?q=artificial+intelligence+OR+LLM+OR+Claude+OR+GPT&hl=en-US&gl=US&ceid=US:en', slow=False, ai_filter=False), NewsFeed(name='Google News · Technology', url='https://news.google.com/rss/headlines/section/topic/TECHNOLOGY?hl=en-US&gl=US&ceid=US:en', slow=False, ai_filter=False), NewsFeed(name='Techmeme', url='https://www.techmeme.com/feed.xml', slow=False, ai_filter=False), NewsFeed(name='HN · Front Page', url='https://hnrss.org/frontpage', slow=False, ai_filter=False), NewsFeed(name='HN · 100+ Points', url='https://hnrss.org/newest?points=100', slow=False, ai_filter=False), NewsFeed(name='Dev.to · AI', url='https://dev.to/feed/tag/ai', slow=False, ai_filter=False), NewsFeed(name='Lobsters', url='https://lobste.rs/rss', slow=False, ai_filter=False))), NewsCategory(id='community', name='社区讨论', icon='💬', feeds=(NewsFeed(name='HN · AI/LLM/Agent', url='https://hnrss.org/newest?q=AI+OR+LLM+OR+Claude+OR+agent+OR+GPT+OR+Gemini', slow=False, ai_filter=False), NewsFeed(name='Reddit · LLM/Claude/ML', url='https://old.reddit.com/r/LocalLLaMA+ClaudeAI+MachineLearning+OpenAI/.rss?limit=25', slow=True, ai_filter=False))))

113
daily/news/feeds_loader.py Normal file
View File

@@ -0,0 +1,113 @@
"""从 config/feeds.yaml 加载 RSS 源(失败时回退内置默认)。"""
from __future__ import annotations
import logging
from functools import lru_cache
from pathlib import Path
from typing import Any
from daily.config import ROOT
from daily.news.feeds_types import NewsCategory, NewsFeed
logger = logging.getLogger(__name__)
_FEEDS_FILE = ROOT / "config" / "feeds.yaml"
def _parse_feed(raw: dict[str, Any]) -> NewsFeed:
return NewsFeed(
name=str(raw.get("name") or "").strip(),
url=str(raw.get("url") or "").strip(),
slow=bool(raw.get("slow")),
ai_filter=bool(raw.get("ai_filter")),
)
def _parse_category(raw: dict[str, Any]) -> NewsCategory | None:
cat_id = str(raw.get("id") or "").strip()
if not cat_id:
return None
feeds_raw = raw.get("feeds") or []
feeds: list[NewsFeed] = []
for item in feeds_raw:
if not isinstance(item, dict):
continue
feed = _parse_feed(item)
if feed.name and feed.url:
feeds.append(feed)
if not feeds:
return None
return NewsCategory(
id=cat_id,
name=str(raw.get("name") or cat_id),
icon=str(raw.get("icon") or "📰"),
feeds=tuple(feeds),
)
def _parse_categories(items: Any) -> tuple[NewsCategory, ...]:
if not isinstance(items, list):
return ()
categories: list[NewsCategory] = []
for raw in items:
if not isinstance(raw, dict):
continue
cat = _parse_category(raw)
if cat:
categories.append(cat)
return tuple(categories)
def _load_yaml() -> dict[str, Any] | None:
if not _FEEDS_FILE.exists():
return None
try:
import yaml
except ImportError:
logger.warning("未安装 PyYAML无法读取 %s", _FEEDS_FILE)
return None
try:
data = yaml.safe_load(_FEEDS_FILE.read_text(encoding="utf-8"))
except OSError as exc:
logger.warning("读取 feeds 配置失败: %s", exc)
return None
except Exception as exc:
logger.warning("解析 feeds.yaml 失败: %s", exc)
return None
return data if isinstance(data, dict) else None
@lru_cache(maxsize=1)
def _yaml_payload() -> dict[str, Any] | None:
return _load_yaml()
def load_categories(region: str) -> tuple[NewsCategory, ...]:
data = _yaml_payload()
if data:
block = data.get(region) or {}
categories = _parse_categories(block.get("categories"))
if categories:
return categories
logger.warning("feeds.yaml 中 %s.categories 为空,使用内置默认", region)
from daily.news import feeds_defaults as defaults
if region == "cn":
return defaults.CN_NEWS_CATEGORIES
return defaults.NEWS_CATEGORIES
def load_cn_title_keywords() -> tuple[str, ...]:
data = _yaml_payload()
if data:
block = data.get("cn") or {}
keywords = block.get("title_keywords")
if isinstance(keywords, list):
cleaned = tuple(str(x).strip() for x in keywords if str(x).strip())
if cleaned:
return cleaned
from daily.news.feeds_defaults import CN_AI_TITLE_KEYWORDS
return CN_AI_TITLE_KEYWORDS

21
daily/news/feeds_types.py Normal file
View File

@@ -0,0 +1,21 @@
"""RSS 源数据结构。"""
from __future__ import annotations
from dataclasses import dataclass
@dataclass(frozen=True)
class NewsFeed:
name: str
url: str
slow: bool = False
ai_filter: bool = False
@dataclass(frozen=True)
class NewsCategory:
id: str
name: str
icon: str
feeds: tuple[NewsFeed, ...]

View File

@@ -17,17 +17,19 @@ import certifi
import httpx
from daily.config import env, env_int, news_summary_limit
from daily.news.content_filter import filter_news_items
from daily.news.feeds import NEWS_CATEGORIES, NewsCategory, NewsFeed
from daily.news.feeds_cn import CN_AI_TITLE_KEYWORDS, CN_NEWS_CATEGORIES
logger = logging.getLogger(__name__)
USER_AGENT = "Mozilla/5.0 (compatible; skills-hot-daily/1.0; +https://skills.sh)"
BROWSER_USER_AGENT = (
# 部分站点(如 InfoQ会拦截含 bot 标识的 UARSS 抓取统一用浏览器 UA
RSS_USER_AGENT = (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/131.0.0.0 Safari/537.36"
)
BROWSER_USER_AGENT = RSS_USER_AGENT
STRIP_HTML = re.compile(r"<[^>]+>")
WS = re.compile(r"\s+")
@@ -286,20 +288,25 @@ def _fetch_one(
client: httpx.Client,
category: NewsCategory,
feed: NewsFeed,
) -> list[dict[str, Any]]:
) -> tuple[list[dict[str, Any]], bool]:
max_attempts = max(1, env_int("DAILY_RSS_RETRY", 3))
last_exc: Exception | None = None
for url in _reddit_fetch_urls(feed.url):
try:
headers = _request_headers(dict(client.headers), url)
resp = client.get(url, headers=headers)
resp.raise_for_status()
entries = _parse_feed(resp.text, feed.name, category)
return _filter_ai_entries(entries, ai_filter=feed.ai_filter)
except Exception as exc:
last_exc = exc
continue
for attempt in range(max_attempts):
try:
headers = _request_headers(dict(client.headers), url)
resp = client.get(url, headers=headers)
resp.raise_for_status()
entries = _parse_feed(resp.text, feed.name, category)
return _filter_ai_entries(entries, ai_filter=feed.ai_filter), True
except Exception as exc:
last_exc = exc
if attempt + 1 < max_attempts:
time.sleep(min(2.0 * (attempt + 1), 5.0))
continue
break
logger.warning("RSS fetch failed [%s] %s: %s", feed.name, feed.url, last_exc)
return []
return [], False
def _dedupe_items(items: list[dict[str, Any]]) -> list[dict[str, Any]]:
@@ -337,14 +344,20 @@ def _fetch_news(categories: tuple[NewsCategory, ...]) -> dict[str, Any]:
per_category = _per_category_limit()
cutoff = _now_utc() - timedelta(hours=hours)
headers = {"User-Agent": USER_AGENT, "Accept": "application/rss+xml, application/atom+xml, application/xml, text/xml, */*"}
headers = {"User-Agent": RSS_USER_AGENT, "Accept": "application/rss+xml, application/atom+xml, application/xml, text/xml, */*"}
tasks: list[tuple[NewsCategory, NewsFeed]] = []
for category in categories:
for feed in category.feeds:
tasks.append((category, feed))
raw_by_category: dict[str, list[dict[str, Any]]] = {c.id: [] for c in categories}
stats = {"feeds_total": len(tasks), "feeds_ok": 0, "items_raw": 0}
stats: dict[str, Any] = {
"feeds_total": len(tasks),
"feeds_ok": 0,
"items_raw": 0,
"feeds_failed": [],
"content_filtered": 0,
}
with httpx.Client(timeout=15.0, verify=certifi.where(), follow_redirects=True, headers=headers) as client:
fast_tasks = [t for t in tasks if not t[1].slow]
@@ -358,19 +371,24 @@ def _fetch_news(categories: tuple[NewsCategory, ...]) -> dict[str, Any]:
for future in as_completed(futures):
cat_id, feed_name = futures[future]
try:
entries = future.result()
entries, ok = future.result()
except Exception as exc:
logger.warning("RSS 任务异常 [%s]: %s", feed_name, exc)
stats["feeds_failed"].append(feed_name)
continue
if entries:
if ok:
stats["feeds_ok"] += 1
else:
stats["feeds_failed"].append(feed_name)
stats["items_raw"] += len(entries)
raw_by_category[cat_id].extend(entries[:per_feed])
for cat, feed in slow_tasks:
entries = _fetch_one(client, cat, feed)
if entries:
entries, ok = _fetch_one(client, cat, feed)
if ok:
stats["feeds_ok"] += 1
else:
stats["feeds_failed"].append(feed.name)
stats["items_raw"] += len(entries)
raw_by_category[cat.id].extend(entries[:per_feed])
if _is_reddit_url(feed.url):
@@ -384,6 +402,9 @@ def _fetch_news(categories: tuple[NewsCategory, ...]) -> dict[str, Any]:
for category in categories:
items = raw_by_category[category.id]
items = [i for i in items if _within_window(i, cutoff)]
items, filtered_count = filter_news_items(items)
if filtered_count:
stats["content_filtered"] = stats.get("content_filtered", 0) + filtered_count
items.sort(key=_sort_key, reverse=True)
items = _dedupe_items(items)[:per_category]
for item in items:
@@ -467,6 +488,12 @@ def _format_news_section(
f"> 近 **{hours}h** · {news.get('stats', {}).get('feeds_ok', 0)}/{news.get('stats', {}).get('feeds_total', 0)} 源可用",
"",
]
failed = news.get("stats", {}).get("feeds_failed") or []
if failed:
preview = "".join(failed[:5])
suffix = "" if len(failed) > 5 else ""
lines.append(f"> ⚠️ {len(failed)} 个源抓取失败:{preview}{suffix}")
lines.append("")
if not categories:
lines.append("*暂无可用条目(网络/RSS 源异常或时间窗口内无更新)。*")
@@ -497,12 +524,20 @@ def _format_news_section(
return lines
def prepare_wecom_news_items(news: dict[str, Any]) -> list[dict[str, Any]]:
def _sorted_flat(news: dict[str, Any]) -> list[dict[str, Any]]:
flat = list(news.get("flat") or [])
flat.sort(
key=lambda item: (float(item.get("score") or 0), _sort_key(item)[1]),
reverse=True,
)
return flat
def prepare_wecom_news_items(news: dict[str, Any], *, limit: int | None = None) -> list[dict[str, Any]]:
if not news.get("enabled"):
return []
limit = _wecom_limit()
flat = _dedupe_items(news.get("flat") or [])
flat.sort(key=_sort_key, reverse=True)
pick_limit = limit or _wecom_limit()
flat = _sorted_flat(news)
preferred = ("media", "newsletter", "official", "community", "research", "developer")
picked: list[dict[str, Any]] = []
seen: set[str] = set()
@@ -513,12 +548,12 @@ def prepare_wecom_news_items(news: dict[str, Any]) -> list[dict[str, Any]]:
continue
picked.append(item)
seen.add(link)
if len(picked) >= limit:
if len(picked) >= pick_limit:
break
if len(picked) >= limit:
if len(picked) >= pick_limit:
break
items: list[dict[str, Any]] = []
for item in picked[:limit]:
for item in picked[:pick_limit]:
items.append(
{
"title": item.get("title", "?"),
@@ -526,17 +561,17 @@ def prepare_wecom_news_items(news: dict[str, Any]) -> list[dict[str, Any]]:
"source_name": item.get("source_name", "?"),
"published_fmt": item.get("published_fmt", ""),
"desc_short": _clean_text(item.get("summary", ""), 36),
"score": item.get("score"),
}
)
return items
def prepare_wecom_cn_news_items(news: dict[str, Any]) -> list[dict[str, Any]]:
def prepare_wecom_cn_news_items(news: dict[str, Any], *, limit: int | None = None) -> list[dict[str, Any]]:
if not news.get("enabled"):
return []
limit = _wecom_cn_limit()
flat = _dedupe_items(news.get("flat") or [])
flat.sort(key=_sort_key, reverse=True)
pick_limit = limit or _wecom_cn_limit()
flat = _sorted_flat(news)
preferred = ("media", "tech", "dev")
picked: list[dict[str, Any]] = []
seen_links: set[str] = set()
@@ -551,23 +586,23 @@ def prepare_wecom_cn_news_items(news: dict[str, Any]) -> list[dict[str, Any]]:
picked.append(item)
seen_links.add(link)
seen_sources.add(source)
if len(picked) >= limit:
if len(picked) >= pick_limit:
break
if len(picked) >= limit:
if len(picked) >= pick_limit:
break
if len(picked) < limit:
if len(picked) < pick_limit:
for item in flat:
link = _normalize_link(item.get("link", ""))
if not link or link in seen_links:
continue
picked.append(item)
seen_links.add(link)
if len(picked) >= limit:
if len(picked) >= pick_limit:
break
items: list[dict[str, Any]] = []
for item in picked[:limit]:
for item in picked[:pick_limit]:
items.append(
{
"title": item.get("title", "?"),
@@ -575,6 +610,7 @@ def prepare_wecom_cn_news_items(news: dict[str, Any]) -> list[dict[str, Any]]:
"source_name": item.get("source_name", "?"),
"published_fmt": item.get("published_fmt", ""),
"desc_short": _clean_text(item.get("summary", ""), 36),
"score": item.get("score"),
}
)
return items

171
daily/news/rank.py Normal file
View File

@@ -0,0 +1,171 @@
"""RSS 新闻去重、打分与排序。"""
from __future__ import annotations
import logging
import re
from datetime import datetime, timezone
from typing import Any
from daily.config import env_int
from daily.delta import find_previous_data
from daily.news.fetch import _entry_datetime, _normalize_link, _normalize_title
logger = logging.getLogger(__name__)
_WS = re.compile(r"\s+")
_TOKEN = re.compile(r"[\w]{2,}", re.UNICODE)
_CJK = re.compile(r"[\u4e00-\u9fff]")
CATEGORY_TIER: dict[str, int] = {
"official": 20,
"developer": 16,
"research": 14,
"media": 12,
"newsletter": 10,
"community": 8,
"tech": 12,
"dev": 10,
}
def title_similarity_threshold() -> float:
raw = env_int("DAILY_NEWS_TITLE_SIM", 55)
return max(0, min(raw, 95)) / 100.0
def _title_tokens(title: str) -> set[str]:
normalized = _normalize_title(title)
tokens = set(_TOKEN.findall(normalized))
cjk = "".join(_CJK.findall(title))
for i in range(max(0, len(cjk) - 1)):
tokens.add(cjk[i : i + 2])
if not tokens and normalized:
tokens.add(normalized)
return tokens
def jaccard_similarity(left: set[str], right: set[str]) -> float:
if not left or not right:
return 0.0
inter = len(left & right)
union = len(left | right)
return inter / union if union else 0.0
def fuzzy_dedupe_by_title(
items: list[dict[str, Any]],
*,
threshold: float | None = None,
) -> list[dict[str, Any]]:
"""按标题相似度合并重复报道,保留 score 更高(或更靠前)的条目。"""
if not items:
return []
limit = threshold if threshold is not None else title_similarity_threshold()
kept: list[dict[str, Any]] = []
kept_tokens: list[set[str]] = []
for item in items:
tokens = _title_tokens(str(item.get("title") or ""))
duplicate_idx: int | None = None
for idx, existing_tokens in enumerate(kept_tokens):
if jaccard_similarity(tokens, existing_tokens) >= limit:
duplicate_idx = idx
break
if duplicate_idx is None:
kept.append(item)
kept_tokens.append(tokens)
continue
existing = kept[duplicate_idx]
if float(item.get("score") or 0) > float(existing.get("score") or 0):
kept[duplicate_idx] = item
kept_tokens[duplicate_idx] = tokens
return kept
def _freshness_points(item: dict[str, Any], *, now: datetime | None = None) -> int:
now = now or datetime.now(timezone.utc)
dt = _entry_datetime(item)
if dt is None:
return 4
age_hours = max(0.0, (now - dt).total_seconds() / 3600.0)
if age_hours <= 6:
return 30
if age_hours <= 24:
return 22
if age_hours <= 72:
return 8
return 0
def _source_tier(item: dict[str, Any]) -> int:
category_id = str(item.get("category_id") or "")
return CATEGORY_TIER.get(category_id, 8)
def _novelty_points(item: dict[str, Any], yesterday_links: set[str]) -> int:
link = _normalize_link(str(item.get("link") or ""))
if link and link in yesterday_links:
return -20
return 5
def score_news_item(
item: dict[str, Any],
*,
yesterday_links: set[str] | None = None,
now: datetime | None = None,
) -> int:
links = yesterday_links or set()
total = _source_tier(item) + _freshness_points(item, now=now) + _novelty_points(item, links)
return max(0, total)
def load_yesterday_news_links(date_str: str) -> set[str]:
baseline = find_previous_data(date_str)
if not baseline:
return set()
_, data = baseline
links: set[str] = set()
for key in ("ai_news", "cn_ai_news"):
for item in data.get(key) or []:
if not isinstance(item, dict):
continue
link = _normalize_link(str(item.get("link") or ""))
if link:
links.add(link)
return links
def apply_news_ranking(news: dict[str, Any], *, date_str: str) -> dict[str, Any]:
"""对 categories / flat 打分、模糊去重并写回 score 字段。"""
if not news.get("enabled"):
return news
yesterday_links = load_yesterday_news_links(date_str)
threshold = title_similarity_threshold()
now = datetime.now(timezone.utc)
categories = news.get("categories") or []
for category in categories:
items = list(category.get("items") or [])
for item in items:
item["score"] = score_news_item(item, yesterday_links=yesterday_links, now=now)
items.sort(key=lambda x: float(x.get("score") or 0), reverse=True)
category["items"] = fuzzy_dedupe_by_title(items, threshold=threshold)
flat: list[dict[str, Any]] = []
for category in categories:
flat.extend(category.get("items") or [])
flat.sort(key=lambda x: float(x.get("score") or 0), reverse=True)
news["flat"] = fuzzy_dedupe_by_title(flat, threshold=threshold)
stats = dict(news.get("stats") or {})
stats["ranked_items"] = len(news["flat"])
stats["yesterday_links"] = len(yesterday_links)
news["stats"] = stats
logger.info(
"新闻排序完成:%d 条 flat昨日链接基准 %d",
len(news["flat"]),
len(yesterday_links),
)
return news

View File

@@ -0,0 +1 @@
"""早报生成流水线子模块。"""

145
daily/pipeline/collect.py Normal file
View File

@@ -0,0 +1,145 @@
"""抓取与结构化输入组装。"""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime
from typing import Any
from daily.agent_workflow import is_agent_mode
from daily.config import env_int
from daily.delta import compare_depth
from daily.github.search import fetch_emerging_repos, fetch_topic_hot_repos
from daily.github.trending import fetch_github_trending
from daily.news.fetch import fetch_ai_news, fetch_cn_ai_news
from daily.news.rank import apply_news_ranking
from daily.report_data import build_llm_input
from daily.skills_board import load_boards
from shared.skills_data import load_feed
@dataclass
class ReportLimits:
trending_n: int
hot_n: int
skill_pool: int
wecom_trending: int
wecom_hot: int
github_limit: int
wecom_github: int
emerging_limit: int
wecom_emerging: int
topic_limit: int
wecom_topic: int
@dataclass
class ReportContext:
feed: dict[str, Any]
date_str: str
time_str: str
updated: str
trending: list[dict[str, Any]]
hot: list[dict[str, Any]]
github_trending: list[dict[str, Any]]
github_emerging: list[dict[str, Any]]
github_topic: list[dict[str, Any]]
topic_name: str
ai_news: dict[str, Any]
cn_ai_news: dict[str, Any]
limits: ReportLimits
wecom_limits: dict[str, int]
llm_input: dict[str, Any]
def resolve_limits() -> ReportLimits:
compare_n = compare_depth()
trending_n = env_int("DAILY_TRENDING_LIMIT", 150)
hot_n = max(env_int("DAILY_HOT_LIMIT", 150), compare_n)
skill_pool = max(10, env_int("DAILY_WECOM_SKILL_POOL", 200))
wecom_trending = env_int("DAILY_WECOM_TRENDING", 10)
wecom_hot = env_int("DAILY_WECOM_HOT", 10)
github_limit = env_int("DAILY_GITHUB_TRENDING_LIMIT", 10)
wecom_github = env_int("DAILY_WECOM_GITHUB_TRENDING", env_int("DAILY_WECOM_REPOS", 10))
emerging_limit = env_int("DAILY_GITHUB_EMERGING_LIMIT", 10)
wecom_emerging = env_int("DAILY_WECOM_GITHUB_EMERGING", 10)
topic_limit = env_int("DAILY_GITHUB_TOPIC_LIMIT", 10)
wecom_topic = env_int("DAILY_WECOM_GITHUB_TOPIC", 10)
return ReportLimits(
trending_n=trending_n,
hot_n=hot_n,
skill_pool=skill_pool,
wecom_trending=wecom_trending,
wecom_hot=wecom_hot,
github_limit=github_limit,
wecom_github=wecom_github,
emerging_limit=emerging_limit,
wecom_emerging=wecom_emerging,
topic_limit=topic_limit,
wecom_topic=wecom_topic,
)
def collect_report_context(now: datetime) -> ReportContext:
limits = resolve_limits()
compare_n = compare_depth()
github_fetch_n = max(limits.github_limit, compare_n, limits.wecom_github)
emerging_fetch_n = max(limits.emerging_limit, compare_n, limits.wecom_emerging)
topic_fetch_n = max(limits.topic_limit, compare_n, limits.wecom_topic)
feed = load_feed(force=True)
date_str = now.strftime("%Y-%m-%d")
time_str = now.strftime("%H:%M") + " (UTC+8)"
updated = (feed.get("updatedAt") or "")[:10]
trending, hot = load_boards(feed, trending_limit=limits.trending_n, hot_limit=limits.hot_n)
github_trending = fetch_github_trending(github_fetch_n)
seen_repos = {r["repo"] for r in github_trending}
github_emerging = fetch_emerging_repos(emerging_fetch_n, exclude=seen_repos)
seen_repos.update(r["repo"] for r in github_emerging)
topic_name, github_topic = fetch_topic_hot_repos(topic_fetch_n, exclude=seen_repos)
ai_news = apply_news_ranking(fetch_ai_news(), date_str=date_str)
cn_ai_news = apply_news_ranking(fetch_cn_ai_news(), date_str=date_str)
wecom_limits = {
"trending": limits.wecom_trending,
"hot": limits.wecom_hot,
"trending_pool": limits.skill_pool,
"hot_pool": limits.skill_pool,
"github": limits.wecom_github,
"emerging": limits.wecom_emerging,
"topic": limits.wecom_topic,
"ai_news": env_int("DAILY_WECOM_AI_NEWS", 10),
"cn_ai_news": env_int("DAILY_WECOM_CN_AI_NEWS", 8),
}
llm_input = build_llm_input(
date_str=date_str,
updated=updated,
trending=trending,
hot=hot,
github_trending=github_trending,
github_emerging=github_emerging,
github_topic=github_topic,
topic_name=topic_name,
ai_news=ai_news,
cn_ai_news=cn_ai_news,
wecom_limits=wecom_limits,
agent_mode=is_agent_mode(),
)
return ReportContext(
feed=feed,
date_str=date_str,
time_str=time_str,
updated=updated,
trending=trending,
hot=hot,
github_trending=github_trending,
github_emerging=github_emerging,
github_topic=github_topic,
topic_name=topic_name,
ai_news=ai_news,
cn_ai_news=cn_ai_news,
limits=limits,
wecom_limits=wecom_limits,
llm_input=llm_input,
)

View File

@@ -0,0 +1,180 @@
"""归档 / 企微格式化与摘要构建。"""
from __future__ import annotations
import re
import xml.etree.ElementTree as ET
from typing import Any
import certifi
import httpx
from daily.config import full_desc_limit, wecom_skill_desc_limit
from daily.github.auth import github_html_headers
from daily.news.fetch import prepare_wecom_cn_news_items, prepare_wecom_news_items
from shared.skills_data import format_installs
from daily.pipeline.snapshot import skill_id
def short_desc(text: str, limit: int = 72) -> str:
text = re.sub(r"\s+", " ", text or "").strip()
if limit <= 0 or len(text) <= limit:
return text
return text[: limit - 3] + "..."
def archive_desc(text: str) -> str:
return short_desc(text, full_desc_limit())
def wecom_desc(text: str, limit: int = 36) -> str:
return short_desc(text, limit)
def prepare_skill_item(item: dict[str, Any], prev_ids: set[str], rank: int) -> dict[str, Any]:
badge = ""
sid = skill_id(item)
if sid not in prev_ids and prev_ids:
badge = "🆕"
elif rank == 1:
badge = "👑"
installs_fmt = item.get("installs_fmt") or format_installs(item.get("installs", 0))
title = item.get("source", "?") if item.get("cluster") else item.get("title", "?")
desc = item.get("wecom_desc") or item.get("description") or item.get("cluster_titles") or ""
limit = wecom_skill_desc_limit()
desc_short = desc if item.get("wecom_desc") or limit <= 0 else wecom_desc(desc, limit)
return {
"title": title,
"source": item.get("source", "?"),
"installs_fmt": installs_fmt,
"link": item.get("link", ""),
"desc_short": desc_short,
"badge": badge,
"cluster": bool(item.get("cluster")),
"cluster_count": item.get("cluster_count"),
"cluster_titles": item.get("cluster_titles"),
}
def prepare_github_item(item: dict[str, Any]) -> dict[str, Any]:
return {**item, "desc_short": wecom_desc(item.get("description", ""), 40)}
def build_highlights(
trending: list[dict[str, Any]],
hot: list[dict[str, Any]],
github_trending: list[dict[str, Any]],
github_emerging: list[dict[str, Any]],
ai_news: dict[str, Any] | None = None,
cn_ai_news: dict[str, Any] | None = None,
) -> list[str]:
points: list[str] = []
if ai_news and ai_news.get("enabled"):
top_news = prepare_wecom_news_items(ai_news)
if top_news:
n0 = top_news[0]
pub = f" · {n0['published_fmt']}" if n0.get("published_fmt") else ""
points.append(
f"🌍 AI 时讯 [{n0['title']}]({n0['link']})`{n0.get('source_name', '?')}`{pub}"
)
elif ai_news.get("flat"):
n0 = ai_news["flat"][0]
pub = f" · {n0['published_fmt']}" if n0.get("published_fmt") else ""
points.append(f"🌍 AI 时讯 [{n0['title']}]({n0['link']})`{n0.get('source_name', '?')}`{pub}")
if cn_ai_news and cn_ai_news.get("enabled"):
top_cn = prepare_wecom_cn_news_items(cn_ai_news)
if top_cn:
n0 = top_cn[0]
pub = f" · {n0['published_fmt']}" if n0.get("published_fmt") else ""
points.append(
f"🇨🇳 国内 AI [{n0['title']}]({n0['link']})`{n0.get('source_name', '?')}`{pub}"
)
elif cn_ai_news.get("flat"):
n0 = cn_ai_news["flat"][0]
pub = f" · {n0['published_fmt']}" if n0.get("published_fmt") else ""
points.append(
f"🇨🇳 国内 AI [{n0['title']}]({n0['link']})`{n0.get('source_name', '?')}`{pub}"
)
if trending:
t0 = trending[0]
points.append(f"📈 Skills 榜首 **{t0.get('title')}**{format_installs(t0.get('installs', 0))}")
if github_trending:
g0 = github_trending[0]
stars = g0.get("stars_today_fmt", "")
total = g0.get("total_stars_fmt", "")
star_hint = f"+{stars} today · " if stars else (f"{total} · " if total else "")
points.append(f"🐙 GitHub Trending [{g0['repo']}]({g0['url']}){star_hint}{g0.get('language', '')}")
if github_emerging:
e0 = github_emerging[0]
points.append(f"🌱 新兴 [{e0['repo']}]({e0['url']})(⭐ {e0.get('total_stars_fmt', '?')}")
elif hot:
h0 = hot[0]
points.append(f"🔥 Skills Hot 榜首 **{h0.get('title')}**1H {format_installs(h0.get('installs', 0))}")
while len(points) < 3 and len(trending) > len(points):
item = trending[len(points)]
points.append(f"✨ **{item.get('title')}** · `{item.get('source')}`")
return points[:3]
def format_github_repo_section(repos: list[dict[str, Any]], *, show_created: bool = False) -> list[str]:
lines: list[str] = []
for i, repo in enumerate(repos, 1):
lang = repo.get("language") or ""
stars_today = repo.get("stars_today_fmt") or ""
total = repo.get("total_stars_fmt") or ""
created = repo.get("created_at") or ""
meta_parts = [lang]
if stars_today:
meta_parts.append(f"+{stars_today} today")
if total:
meta_parts.append(f"总 ⭐ {total}")
if show_created and created:
meta_parts.append(f"创建于 {created}")
lines.append(f"{i}. **[{repo['repo']}]({repo['url']})** · {' · '.join(meta_parts)}")
desc = archive_desc(repo.get("description", ""))
if desc:
lines.append(f" - {desc}")
lines.append("")
return lines
def format_skill_section(items: list[dict[str, Any]], *, hot: bool = False) -> list[str]:
lines: list[str] = []
for i, item in enumerate(items, 1):
sid = item.get("id") or f"{item.get('source', '?')}/{item.get('title', '?')}"
link = item.get("link", "")
installs = format_installs(item.get("installs", 0))
meta = f"1H {installs}" if hot else f"总安装 {installs}"
if link:
lines.append(f"{i}. **[{sid}]({link})** · {meta}")
else:
lines.append(f"{i}. **{sid}** · {meta}")
desc = archive_desc(item.get("description", ""))
if desc:
lines.append(f" - {desc}")
lines.append("")
return lines
def fetch_latest_release_title(repo: str) -> str | None:
atom_url = f"https://github.com/{repo}/releases.atom"
try:
with httpx.Client(
timeout=12.0,
verify=certifi.where(),
follow_redirects=True,
headers=github_html_headers(),
) as client:
resp = client.get(atom_url)
if resp.status_code != 200:
return None
root = ET.fromstring(resp.text)
ns = {"a": "http://www.w3.org/2005/Atom"}
entry = root.find("a:entry", ns)
if entry is None:
return None
title = entry.find("a:title", ns)
return title.text.strip() if title is not None and title.text else None
except Exception:
return None

116
daily/pipeline/localize.py Normal file
View File

@@ -0,0 +1,116 @@
"""归档内容中文化。"""
from __future__ import annotations
from typing import Any
from daily.config import full_desc_limit, news_summary_limit
from daily.localize import LocalizeJob, localize_descriptions, needs_chinese
from daily.pipeline.snapshot import skill_id
def localize_descriptions_in_place(
trending: list[dict[str, Any]],
hot: list[dict[str, Any]],
github_trending: list[dict[str, Any]],
github_emerging: list[dict[str, Any]],
github_topic: list[dict[str, Any]],
ai_news: dict[str, Any],
cn_ai_news: dict[str, Any] | None = None,
) -> None:
full_limit = full_desc_limit()
news_limit = news_summary_limit()
jobs: list[LocalizeJob] = []
seen_skill: set[str] = set()
for item in trending + hot:
sid = skill_id(item)
if sid in seen_skill:
continue
seen_skill.add(sid)
desc = (item.get("description") or "").strip()
if desc:
jobs.append(LocalizeJob(f"skill:{sid}", desc, full_limit))
seen_repo: set[str] = set()
for repo_list in (github_trending, github_emerging, github_topic):
for item in repo_list:
repo = item.get("repo", "")
if not repo or repo in seen_repo:
continue
seen_repo.add(repo)
desc = (item.get("description") or "").strip()
if desc:
jobs.append(LocalizeJob(f"github:{repo}", desc, full_limit))
if ai_news.get("enabled"):
seen_news: set[str] = set()
for item in ai_news.get("flat") or []:
link = item.get("link", "")
if not link or link in seen_news:
continue
seen_news.add(link)
summary = (item.get("summary") or "").strip()
if summary:
jobs.append(LocalizeJob(f"news:{link}", summary, news_limit))
zh_map = localize_descriptions(jobs, archive=True)
if not zh_map and not jobs:
return
def _apply_zh(mapping: dict[str, str]) -> None:
for item in trending + hot:
key = f"skill:{skill_id(item)}"
if key in mapping:
item["description"] = mapping[key]
for repo_list in (github_trending, github_emerging, github_topic):
for item in repo_list:
key = f"github:{item.get('repo', '')}"
if key in mapping:
item["description"] = mapping[key]
if ai_news.get("enabled"):
for cat in ai_news.get("categories") or []:
for item in cat.get("items") or []:
key = f"news:{item.get('link', '')}"
if key in mapping:
item["summary"] = mapping[key]
for item in ai_news.get("flat") or []:
key = f"news:{item.get('link', '')}"
if key in mapping:
item["summary"] = mapping[key]
_apply_zh(zh_map)
retry_jobs: list[LocalizeJob] = []
seen_skill.clear()
for item in trending + hot:
sid = skill_id(item)
if sid in seen_skill:
continue
seen_skill.add(sid)
desc = (item.get("description") or "").strip()
if needs_chinese(desc):
retry_jobs.append(LocalizeJob(f"skill:{sid}", desc, full_limit))
seen_repo.clear()
for repo_list in (github_trending, github_emerging, github_topic):
for item in repo_list:
repo = item.get("repo", "")
if not repo or repo in seen_repo:
continue
seen_repo.add(repo)
desc = (item.get("description") or "").strip()
if needs_chinese(desc):
retry_jobs.append(LocalizeJob(f"github:{repo}", desc, full_limit))
if ai_news.get("enabled"):
seen_news.clear()
for item in ai_news.get("flat") or []:
link = item.get("link", "")
if not link or link in seen_news:
continue
seen_news.add(link)
summary = (item.get("summary") or "").strip()
if needs_chinese(summary):
retry_jobs.append(LocalizeJob(f"news:{link}", summary, news_limit))
if retry_jobs:
_apply_zh(localize_descriptions(retry_jobs, archive=True))

View File

@@ -0,0 +1,36 @@
"""Skills 快照读写。"""
from __future__ import annotations
import json
from typing import Any
from daily.config import CACHE_DIR, SNAPSHOT_FILE
def skill_id(item: dict[str, Any]) -> str:
return str(item.get("id") or f"{item.get('source')}/{item.get('title')}")
def load_snapshot() -> set[str]:
if not SNAPSHOT_FILE.exists():
return set()
try:
data = json.loads(SNAPSHOT_FILE.read_text(encoding="utf-8"))
return set(str(x) for x in (data.get("skill_ids") or []))
except (OSError, json.JSONDecodeError):
return set()
def save_snapshot(feed: dict[str, Any], date_str: str) -> None:
CACHE_DIR.mkdir(parents=True, exist_ok=True)
ids: list[str] = []
for board in ("topTrending", "topHot"):
for item in feed.get(board, [])[:20]:
sid = skill_id(item)
if sid not in ids:
ids.append(sid)
SNAPSHOT_FILE.write_text(
json.dumps({"date": date_str, "skill_ids": ids}, ensure_ascii=False, indent=2),
encoding="utf-8",
)

53
daily/pipeline/themes.py Normal file
View File

@@ -0,0 +1,53 @@
"""主题检测与聚类。"""
from __future__ import annotations
from collections import defaultdict
from typing import Any
THEME_RULES: list[tuple[str, str, list[str]]] = [
("🎬", "AI 多媒体 / 视频", ["runcomfy", "remotion", "video", "seedance", "inpaint", "lipsync"]),
("🔧", "工程协作 / Skill 元能力", ["grill", "tdd", "architecture", "find-skills", "to-issues"]),
("📱", "飞书 / Lark", ["lark", "feishu"]),
("📣", "内容营销", ["viral", "tiktok", "instagram", "reels"]),
("🎨", "设计 / 前端", ["frontend", "design", "ui-ux", "tailwind"]),
]
def detect_theme_line(feed: dict[str, Any]) -> str:
scores: dict[str, int] = defaultdict(int)
for board in ("topTrending", "topHot"):
for rank, item in enumerate(feed.get(board, [])[:10], 1):
haystack = " ".join(
[item.get("title", ""), item.get("source", ""), item.get("description", "")]
).lower()
for _icon, label, keywords in THEME_RULES:
if any(k in haystack for k in keywords):
scores[label] += max(1, 11 - rank)
break
if not scores:
return "**今日主题**Agent Skills 生态持续活跃"
return f"**今日主题**{max(scores.items(), key=lambda x: x[1])[0]}"
def theme_clusters(feed: dict[str, Any], limit: int = 5) -> list[tuple[str, list[str]]]:
from daily.pipeline.snapshot import skill_id
buckets: dict[str, list[str]] = defaultdict(list)
seen: set[str] = set()
for board in ("topTrending", "topHot"):
for item in feed.get(board, [])[:20]:
item_id = skill_id(item)
if item_id in seen:
continue
seen.add(item_id)
haystack = " ".join(
[item.get("title", ""), item.get("source", ""), item.get("description", "")]
).lower()
for _icon, theme, keywords in THEME_RULES:
if any(k in haystack for k in keywords):
label = f"**{item.get('title')}** (`{item.get('source')}`)"
if label not in buckets[theme]:
buckets[theme].append(label)
break
return [(theme, examples[:limit]) for theme, examples in buckets.items() if examples]

View File

@@ -61,33 +61,47 @@ def _slim_news_items(
prepare=prepare_wecom_news_items,
) -> list[dict[str, Any]]:
items: list[dict[str, Any]] = []
for item in prepare(ai_news):
items.append(
{
"link": item.get("link", ""),
"title": item.get("title", ""),
"source_name": item.get("source_name", ""),
"published_fmt": item.get("published_fmt", ""),
"summary": item.get("desc_short") or "",
}
)
for item in prepare(ai_news, limit=limit):
payload = {
"link": item.get("link", ""),
"title": item.get("title", ""),
"source_name": item.get("source_name", ""),
"published_fmt": item.get("published_fmt", ""),
"summary": item.get("desc_short") or "",
}
if item.get("score") is not None:
payload["score"] = item.get("score")
items.append(payload)
if len(items) >= limit:
break
if items:
return items
for item in (ai_news.get("flat") or [])[:limit]:
items.append(
{
"link": item.get("link", ""),
"title": item.get("title", ""),
"source_name": item.get("source_name", ""),
"published_fmt": item.get("published_fmt", ""),
"summary": item.get("summary", ""),
}
)
flat = sorted(
ai_news.get("flat") or [],
key=lambda row: float(row.get("score") or 0),
reverse=True,
)
for item in flat[:limit]:
payload = {
"link": item.get("link", ""),
"title": item.get("title", ""),
"source_name": item.get("source_name", ""),
"published_fmt": item.get("published_fmt", ""),
"summary": item.get("summary", ""),
}
if item.get("score") is not None:
payload["score"] = item.get("score")
items.append(payload)
return items
def _news_pool_limit(wecom_limit: int, *, env_key: str, default_pool: int, agent_mode: bool) -> int:
if not agent_mode:
return wecom_limit
pool = env_int(env_key, default_pool)
return max(wecom_limit, pool)
def _wecom_skill_pool() -> int:
return max(10, env_int("DAILY_WECOM_SKILL_POOL", 200))
@@ -105,10 +119,21 @@ def build_llm_input(
ai_news: dict[str, Any],
cn_ai_news: dict[str, Any],
wecom_limits: dict[str, int],
agent_mode: bool = False,
) -> dict[str, Any]:
"""供 Cursor 编辑的精简 JSON不含完整 markdown"""
news_limit = wecom_limits.get("ai_news", 10)
cn_news_limit = wecom_limits.get("cn_ai_news", 8)
news_limit = _news_pool_limit(
wecom_limits.get("ai_news", 10),
env_key="DAILY_AGENT_NEWS_POOL",
default_pool=40,
agent_mode=agent_mode,
)
cn_news_limit = _news_pool_limit(
wecom_limits.get("cn_ai_news", 8),
env_key="DAILY_AGENT_CN_NEWS_POOL",
default_pool=30,
agent_mode=agent_mode,
)
depth = compare_depth()
trend_cmp = trending[:depth]
hot_cmp = hot[:depth]

View File

@@ -15,7 +15,7 @@ logger = logging.getLogger(__name__)
Board = Literal["trending", "hot"]
SKILLS_SITE = "https://www.skills.sh"
USER_AGENT = "Mozilla/5.0 (compatible; skills-hot-daily/1.0; +https://skills.sh)"
USER_AGENT = "Mozilla/5.0 (compatible; daily-robots/1.0; +https://skills.sh)"
_SKILL_RE = re.compile(
r'\{"source":"(?P<source>[^"]+)","skillId":"(?P<skill_id>[^"]+)",'

14
docker-compose.yml Normal file
View File

@@ -0,0 +1,14 @@
services:
daily:
build: .
env_file: .env
environment:
TZ: Asia/Shanghai
volumes:
- ./output:/app/output
- ./.cache:/app/.cache
- ./logs:/app/logs
# Examples:
# docker compose run --rm daily
# docker compose run --rm daily --skip-push
# docker compose run --rm daily --force

View File

@@ -0,0 +1,35 @@
<!-- 示例Agent 模式企微早报节选,日期 2026-07-03 -->
📰 **早报 · 2026-07-03**
> ⏱ 09:30 (UTC+8) · 数据截至 2026-07-02
> remotion-render 以 21549 次安装稳坐 Skills Trending 榜首halt-catch-fire 五件套把「TSX 直出 MP4」推到台前。GitHub 侧 openclaw 仍以 381.5K star 居 Trending 第一;新闻头条则是 OpenAI 拟向特朗普政府让渡 AI 繁荣 5% 分成——多媒体 skill 装机、本地 Agent 开源栈与 AI 治理议题同日撞屏。
🎯 **代码出片霸榜Agent基建共振**
💡 **今日信号**
> 🔥 Skills Hotxixu-me/skills 12 件套新占 Top10 前十,含 xdrop、openclaw-secure-linux-cloud 等开发者工具簇
> 📦 Skills较昨日 Trending Top15 新入 7 条,飞书 lark-approval、lark-wiki 等办公 skill 批量涌入
> 🐙 GitHubaffaan-m/ECC 225.2K star 新入 Trending #2面向 Claude Code/Codex/Cursor 的 Agent harness 优化成显学
> 🌱 GitHublangflow、dify 新入 Trending Top10可视化 Agent 工作流平台与 DeepSpec 推测解码基建各据一角
> 📰 时讯Anthropic 搁置已久的 Fable 5 获准回归,模型内容政策与 Zuckerberg「Agent 进展慢于预期」形成对照
📦 **今日首推**
`npx skills add halt-catch-fire/skills/remotion-render`
> Trending #1、21549 安装5 skill 集群覆盖 remotion-render 到 ai-video-generation把 React/Remotion 组件程序化渲染 MP4 做成默认可装工作流。
🌍 **国际 AI · 精选 10**
1. [OpenAI拟向特朗普政府让渡AI繁荣5%分成](https://www.theverge.com/ai-artificial-intelligence/960588/openai-government-5-percent-stake-trump) — The Verge 头条:头部模型公司探讨向政府让渡 AI 收益分成,商业化与治理边界被推上舆论前台
2. [Anthropic搁置已久的Fable 5获准回归](https://www.theverge.com/ai-artificial-intelligence/958964/anthropic-claude-fable-5-is-back) — 经数周协商后,此前被搁置的 Claude Fable 5 内容政策模型重新获准上线
3. [Zuckerberg称AI Agent进展慢于预期](https://techcrunch.com/2026/07/02/mark-zuckerberg-tells-staff-that-ai-agents-havent-progressed-as-quickly-as-hed-hoped/) — Meta 内部会议透露Agent 落地节奏未达此前预期
🇨🇳 **国内 AI · 精选 8**
1. [让Agent越用越强AReaL 2.0开源打造面向自演进智能体的RL基础设施](https://www.qbitai.com/2026/07/442134.html) — 量子位:面向自演进 Agent 的 RL 训练基建开源
2. [Agent Loop 深度调研:把决定权交给模型的一次换代,为什么发生在现在](https://juejin.cn/post/7657771483503018036) — 掘金长文梳理 Agent Loop 架构演进
📈 **Skills Trending Top 9**
1. [**halt-catch-fire/skills** · 5 skills · **21.5K21.5K**](https://www.skills.sh/halt-catch-fire/skills/remotion-render)
> 用 inference.sh 将 Remotion 组件代码直接渲染为 MP4可配分辨率帧率时长适合程序化视
2. [**find-skills**](https://www.skills.sh/vercel-labs/skills/find-skills) · `vercel-labs/skills` · **18.0K**
> 用户询问如何做某事或寻找能力扩展时,帮助发现并安装合适的 Agent Skill。
<!-- 完整版还包含 Hot 榜、GitHub Trending/新兴/Topic 等分区,见本地 output/ -->

3
pytest.ini Normal file
View File

@@ -0,0 +1,3 @@
[pytest]
testpaths = tests
pythonpath = .

4
requirements-dev.txt Normal file
View File

@@ -0,0 +1,4 @@
python-dotenv>=1.0.0
httpx>=0.27.0
certifi>=2024.0.0
pytest>=8.0

View File

@@ -1,5 +1,6 @@
python-dotenv>=1.0.0
httpx>=0.27.0
certifi>=2024.0.0
PyYAML>=6.0.0
# 英文描述转中文(使用 CURSOR_API_KEY 时需安装)
cursor-sdk>=0.1.0

View File

@@ -3,16 +3,21 @@
# .\run-daily.ps1
# .\run-daily.ps1 -SkipPush
# .\run-daily.ps1 -SkipGenerate
# .\run-daily.ps1 -Force # bypass duplicate-run lock
param(
[switch]$SkipPush,
[switch]$SkipGenerate
[switch]$SkipGenerate,
[switch]$Force
)
$ErrorActionPreference = "Stop"
$Root = Split-Path -Parent $MyInvocation.MyCommand.Path
Set-Location $Root
$LockFile = Join-Path $Root ".cache\run-daily.lock"
$LockMaxMinutes = 30
function Import-DotEnvFile {
param([string]$Path)
if (-not (Test-Path $Path)) { return }
@@ -29,30 +34,61 @@ function Import-DotEnvFile {
}
}
Import-DotEnvFile (Join-Path $Root ".env")
Import-DotEnvFile (Join-Path $Root ".env.local")
function Test-RunDailyLock {
if (-not (Test-Path $LockFile)) { return $false }
$age = (Get-Date) - (Get-Item $LockFile).LastWriteTime
return $age.TotalMinutes -lt $LockMaxMinutes
}
$python = "python"
$date = Get-Date -Format "yyyy-MM-dd"
$reportWecom = Join-Path $Root "output\$date.wecom.md"
function Set-RunDailyLock {
$dir = Split-Path $LockFile -Parent
if (-not (Test-Path $dir)) {
New-Item -ItemType Directory -Path $dir -Force | Out-Null
}
Set-Content -Path $LockFile -Value (Get-Date -Format "o") -Encoding UTF8
}
if (-not $SkipGenerate) {
Write-Host "Generating daily report: $date"
& $python -m daily generate
if ($LASTEXITCODE -ne 0) {
throw "daily generate failed with exit code $LASTEXITCODE"
function Clear-RunDailyLock {
if (Test-Path $LockFile) {
Remove-Item $LockFile -Force -ErrorAction SilentlyContinue
}
}
if (-not $SkipPush) {
if (-not (Test-Path $reportWecom)) {
throw "Report not found: $reportWecom"
}
Write-Host "Pushing to WeCom webhook..."
& $python -m daily push $reportWecom
if ($LASTEXITCODE -ne 0) {
throw "daily push failed with exit code $LASTEXITCODE"
}
if (-not $Force -and (Test-RunDailyLock)) {
Write-Host "Skip: run-daily already ran within ${LockMaxMinutes} minutes (lock: $LockFile). Use -Force to override."
exit 0
}
Write-Host "Done: $date"
Set-RunDailyLock
try {
Import-DotEnvFile (Join-Path $Root ".env")
Import-DotEnvFile (Join-Path $Root ".env.local")
$python = "python"
$date = Get-Date -Format "yyyy-MM-dd"
$reportWecom = Join-Path $Root "output\$date.wecom.md"
if (-not $SkipGenerate) {
Write-Host "Generating daily report: $date"
& $python -m daily generate
if ($LASTEXITCODE -ne 0) {
throw "daily generate failed with exit code $LASTEXITCODE"
}
}
if (-not $SkipPush) {
if (-not (Test-Path $reportWecom)) {
throw "Report not found: $reportWecom"
}
Write-Host "Pushing to WeCom webhook..."
& $python -m daily push $reportWecom
if ($LASTEXITCODE -ne 0) {
throw "daily push failed with exit code $LASTEXITCODE"
}
}
Write-Host "Done: $date"
}
finally {
Clear-RunDailyLock
}

106
run-daily.sh Normal file
View File

@@ -0,0 +1,106 @@
#!/usr/bin/env bash
# Daily report: generate + push to WeCom webhook
# Usage:
# ./run-daily.sh
# ./run-daily.sh --skip-push
# ./run-daily.sh --skip-generate
# ./run-daily.sh --force
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$ROOT"
LOCK_FILE="$ROOT/.cache/run-daily.lock"
LOCK_MAX_MINUTES=30
SKIP_PUSH=0
SKIP_GENERATE=0
FORCE=0
while [[ $# -gt 0 ]]; do
case "$1" in
--skip-push) SKIP_PUSH=1 ;;
--skip-generate) SKIP_GENERATE=1 ;;
--force) FORCE=1 ;;
-h|--help)
echo "Usage: $0 [--skip-push] [--skip-generate] [--force]"
exit 0
;;
*)
echo "Unknown option: $1" >&2
exit 1
;;
esac
shift
done
load_dotenv() {
local file="$1"
[[ -f "$file" ]] || return 0
while IFS= read -r line || [[ -n "$line" ]]; do
[[ "$line" =~ ^[[:space:]]*# ]] && continue
[[ "$line" != *"="* ]] && continue
local name="${line%%=*}"
local value="${line#*=}"
name="${name#"${name%%[![:space:]]*}"}"
name="${name%"${name##*[![:space:]]}"}"
value="${value#"${value%%[![:space:]]*}"}"
value="${value%"${value##*[![:space:]]}"}"
value="${value%\"}"
value="${value#\"}"
value="${value%\'}"
value="${value#\'}"
if [[ -n "$name" && -n "$value" ]]; then
export "$name=$value"
fi
done < "$file"
}
lock_active() {
[[ -f "$LOCK_FILE" ]] || return 1
local now lock_ts age_minutes
now="$(date +%s)"
lock_ts="$(date -r "$LOCK_FILE" +%s 2>/dev/null || stat -c %Y "$LOCK_FILE" 2>/dev/null || echo 0)"
age_minutes=$(( (now - lock_ts) / 60 ))
[[ "$age_minutes" -lt "$LOCK_MAX_MINUTES" ]]
}
set_lock() {
mkdir -p "$(dirname "$LOCK_FILE")"
date -Iseconds > "$LOCK_FILE"
}
clear_lock() {
rm -f "$LOCK_FILE"
}
if [[ "$FORCE" -eq 0 ]] && lock_active; then
echo "Skip: run-daily already ran within ${LOCK_MAX_MINUTES} minutes (lock: $LOCK_FILE). Use --force to override."
exit 0
fi
set_lock
trap clear_lock EXIT
load_dotenv "$ROOT/.env"
load_dotenv "$ROOT/.env.local"
PYTHON="${PYTHON:-python}"
DATE="$(TZ=Asia/Shanghai date +%Y-%m-%d)"
REPORT_WECOM="$ROOT/output/${DATE}.wecom.md"
if [[ "$SKIP_GENERATE" -eq 0 ]]; then
echo "Generating daily report: $DATE"
"$PYTHON" -m daily generate
fi
if [[ "$SKIP_PUSH" -eq 0 ]]; then
if [[ ! -f "$REPORT_WECOM" ]]; then
echo "Report not found: $REPORT_WECOM" >&2
exit 1
fi
echo "Pushing to WeCom webhook..."
"$PYTHON" -m daily push "$REPORT_WECOM"
fi
echo "Done: $DATE"

View File

@@ -0,0 +1,47 @@
from pathlib import Path
import yaml
from daily.news.feeds_types import NewsCategory, NewsFeed
data = yaml.safe_load(Path("config/feeds.yaml").read_text(encoding="utf-8"))
def parse_feed(raw: dict) -> NewsFeed:
return NewsFeed(
name=raw["name"],
url=raw["url"],
slow=bool(raw.get("slow")),
ai_filter=bool(raw.get("ai_filter")),
)
def parse_cat(raw: dict) -> NewsCategory:
return NewsCategory(
id=raw["id"],
name=raw["name"],
icon=raw["icon"],
feeds=tuple(parse_feed(item) for item in raw["feeds"]),
)
intl = tuple(parse_cat(item) for item in data["intl"]["categories"])
cn = tuple(parse_cat(item) for item in data["cn"]["categories"])
keywords = tuple(data["cn"]["title_keywords"])
lines = [
'"""Built-in RSS defaults when config/feeds.yaml is missing or invalid."""',
"",
"from __future__ import annotations",
"",
"from daily.news.feeds_types import NewsCategory, NewsFeed",
"",
f"CN_AI_TITLE_KEYWORDS: tuple[str, ...] = {keywords!r}",
"",
f"CN_NEWS_CATEGORIES: tuple[NewsCategory, ...] = {cn!r}",
"",
f"NEWS_CATEGORIES: tuple[NewsCategory, ...] = {intl!r}",
"",
]
Path("daily/news/feeds_defaults.py").write_text("\n".join(lines), encoding="utf-8")
print("written")

10
shared/__init__.py Normal file
View File

@@ -0,0 +1,10 @@
"""Repo-root shared modules (skills feed data, etc.)."""
from shared.skills_data import (
Board,
format_installs,
load_feed,
warm_feed_cache,
)
__all__ = ["Board", "format_installs", "load_feed", "warm_feed_cache"]

109
shared/skills_data.py Normal file
View File

@@ -0,0 +1,109 @@
"""skills.sh feed.json 拉取、缓存与榜单数据。"""
from __future__ import annotations
import json
import logging
import time
from pathlib import Path
from typing import Any, Literal
import certifi
import httpx
logger = logging.getLogger(__name__)
ROOT = Path(__file__).resolve().parent.parent
FEED_URLS = [
"https://cdn.jsdelivr.net/gh/NeverSight/skills.sh_feed@main/data/feed.json",
"https://raw.githubusercontent.com/NeverSight/skills.sh_feed/main/data/feed.json",
]
CACHE_TTL_SECONDS = 600
CACHE_DIR = ROOT / ".cache"
CACHE_FILE = CACHE_DIR / "skills-feed.json"
USER_AGENT = "Mozilla/5.0 (compatible; daily-robots/1.0; +https://skills.sh)"
_cache: dict[str, Any] = {"data": None, "fetched_at": 0.0}
Board = Literal["trending", "hot", "all"]
BOARD_FEED_KEYS = {
"trending": "topTrending",
"hot": "topHot",
"all": "topAllTime",
}
def _fetch_json(url: str) -> dict[str, Any]:
headers = {"User-Agent": USER_AGENT, "Accept": "application/json"}
with httpx.Client(
timeout=httpx.Timeout(20.0, connect=10.0),
verify=certifi.where(),
follow_redirects=True,
) as client:
resp = client.get(url, headers=headers)
resp.raise_for_status()
return resp.json()
def _load_disk_cache() -> dict[str, Any] | None:
if not CACHE_FILE.exists():
return None
try:
return json.loads(CACHE_FILE.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
logger.warning("读取 skills feed 本地缓存失败: %s", exc)
return None
def _save_disk_cache(data: dict[str, Any]) -> None:
CACHE_DIR.mkdir(parents=True, exist_ok=True)
CACHE_FILE.write_text(json.dumps(data, ensure_ascii=False), encoding="utf-8")
def load_feed(force: bool = False) -> dict[str, Any]:
now = time.time()
if not force and _cache["data"] and now - _cache["fetched_at"] < CACHE_TTL_SECONDS:
return _cache["data"]
errors: list[str] = []
for url in FEED_URLS:
for attempt in range(3):
try:
data = _fetch_json(url)
_cache["data"] = data
_cache["fetched_at"] = now
_save_disk_cache(data)
logger.info("skills 数据已更新: %s", url)
return data
except Exception as exc:
msg = f"{url} (#{attempt + 1}): {exc}"
errors.append(msg)
logger.debug("拉取失败 %s", msg)
time.sleep(0.5 * (attempt + 1))
stale = _load_disk_cache()
if stale:
logger.warning("网络不可用,回退到本地 skills feed 缓存")
_cache["data"] = stale
_cache["fetched_at"] = now
return stale
raise RuntimeError(f"无法获取 skills 数据。最近错误: {errors[-1]}")
def warm_feed_cache() -> None:
"""启动时预加载,避免首条消息才触发网络请求。"""
load_feed(force=True)
def format_installs(n: int | float) -> str:
if n >= 1_000_000:
return f"{n / 1_000_000:.1f}M"
if n >= 1_000:
return f"{n / 1_000:.1f}K"
return str(int(n))
def board_items(feed: dict[str, Any], board: Board) -> list[dict[str, Any]]:
return feed.get(BOARD_FEED_KEYS[board], [])

50
tests/conftest.py Normal file
View File

@@ -0,0 +1,50 @@
from __future__ import annotations
import json
from datetime import datetime, timedelta, timezone
from pathlib import Path
import pytest
FIXTURES = Path(__file__).parent / "fixtures"
@pytest.fixture
def skills_feed() -> dict:
return json.loads((FIXTURES / "skills-feed.json").read_text(encoding="utf-8"))
@pytest.fixture
def fixed_cst():
return datetime(2026, 7, 3, 9, 30, tzinfo=timezone(timedelta(hours=8)))
@pytest.fixture
def isolated_output(tmp_path, monkeypatch):
import daily.config as config
import daily.report_data as report_data
out = tmp_path / "output"
cache = tmp_path / "cache"
logs = tmp_path / "logs"
out.mkdir()
cache.mkdir()
logs.mkdir()
monkeypatch.setattr(config, "OUTPUT_DIR", out)
monkeypatch.setattr(report_data, "OUTPUT_DIR", out)
monkeypatch.setattr(config, "CACHE_DIR", cache)
monkeypatch.setattr(config, "LOG_DIR", logs)
monkeypatch.setattr(config, "SNAPSHOT_FILE", cache / "last-report.json")
return out
@pytest.fixture
def offline_generate_env(monkeypatch):
monkeypatch.setenv("DAILY_AI_NEWS", "0")
monkeypatch.setenv("DAILY_CN_AI_NEWS", "0")
monkeypatch.setenv("DAILY_ZH_DESC", "0")
monkeypatch.setenv("SKILLS_BOARD_SOURCE", "feed")
monkeypatch.setenv("DAILY_REPORT_MODE", "classic")
monkeypatch.setenv("DAILY_CURSOR_EDITOR", "0")
monkeypatch.setenv("DAILY_TRENDING_LIMIT", "10")
monkeypatch.setenv("DAILY_HOT_LIMIT", "10")

12
tests/fixtures/sample-rss.xml vendored Normal file
View File

@@ -0,0 +1,12 @@
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0">
<channel>
<title>Sample AI Feed</title>
<item>
<title>Sample AI headline for smoke tests</title>
<link>https://example.com/ai-news/1</link>
<pubDate>Wed, 02 Jul 2026 08:00:00 GMT</pubDate>
<description>A minimal RSS item used by offline tests.</description>
</item>
</channel>
</rss>

32
tests/fixtures/skills-feed.json vendored Normal file
View File

@@ -0,0 +1,32 @@
{
"updatedAt": "2026-07-02T12:00:00Z",
"topTrending": [
{
"id": "vercel-labs/skills/find-skills",
"title": "find-skills",
"source": "vercel-labs/skills",
"installs": 18000,
"link": "https://www.skills.sh/vercel-labs/skills/find-skills",
"description": "Help users discover and install Agent Skills."
},
{
"id": "halt-catch-fire/skills/remotion-render",
"title": "remotion-render",
"source": "halt-catch-fire/skills",
"installs": 21549,
"link": "https://www.skills.sh/halt-catch-fire/skills/remotion-render",
"description": "Render videos from React/Remotion component code."
}
],
"topHot": [
{
"id": "vercel-labs/skills/find-skills",
"title": "find-skills",
"source": "vercel-labs/skills",
"installs": 243,
"link": "https://www.skills.sh/vercel-labs/skills/find-skills",
"description": "Help users discover and install Agent Skills."
}
],
"topAllTime": []
}

View File

@@ -0,0 +1,85 @@
from __future__ import annotations
import json
def test_analyze_trends_parses_json(monkeypatch, isolated_output):
import daily.agent_workflow as agent
import daily.config as config
monkeypatch.setattr(config, "OUTPUT_DIR", isolated_output)
monkeypatch.setattr(agent, "OUTPUT_DIR", isolated_output)
monkeypatch.setattr(
agent,
"llm_chat",
lambda system, user: json.dumps(
{
"headline": "Skills 视频工具升温",
"opening": "今日 remotion 相关技能继续走强。",
"themes": [{"name": "视频", "summary": "Remotion 生态活跃"}],
"top_picks": [],
"signals": [],
},
ensure_ascii=False,
),
)
llm_input = {"date": "2026-07-03", "skills_trending": []}
trends = agent.analyze_trends(llm_input, date_str="2026-07-03")
assert trends is not None
assert trends["headline"] == "Skills 视频工具升温"
assert (isolated_output / "2026-07-03.trends.json").exists()
def test_write_wecom_report_extracts_markdown_block(monkeypatch):
import daily.agent_workflow as agent
monkeypatch.setattr(
agent,
"llm_chat",
lambda system, user: "```markdown\n📰 **早报 · 2026-07-03**\n\n正文\n```",
)
md = agent.write_wecom_report(
{"date": "2026-07-03"},
{"headline": "test", "opening": "open"},
date_str="2026-07-03",
time_str="09:30 (UTC+8)",
updated="2026-07-02",
)
assert md is not None
assert md.startswith("📰")
assert "正文" in md
def test_run_agent_workflow_retries_step2(monkeypatch):
import daily.agent_workflow as agent
calls = {"n": 0}
def fake_llm(system, user):
if '"trends"' in user:
calls["n"] += 1
if calls["n"] == 1:
raise RuntimeError("Bridge request timed out")
return "📰 **早报 · 2026-07-03**\n\n重试成功"
return json.dumps(
{"headline": "h", "opening": "o", "themes": [], "top_picks": [], "signals": []},
ensure_ascii=False,
)
monkeypatch.setattr(agent, "llm_chat", fake_llm)
md = agent.run_agent_workflow(
{"date": "2026-07-03"},
date_str="2026-07-03",
time_str="09:30 (UTC+8)",
updated="2026-07-02",
)
assert md is not None
assert "重试成功" in md
assert calls["n"] == 2

View File

@@ -0,0 +1,15 @@
from __future__ import annotations
def test_cursor_timeout_seconds_default(monkeypatch):
import daily.cursor_client as cc
monkeypatch.delenv("CURSOR_TIMEOUT", raising=False)
assert cc.cursor_timeout_seconds() == 600.0
def test_cursor_timeout_seconds_from_env(monkeypatch):
import daily.cursor_client as cc
monkeypatch.setenv("CURSOR_TIMEOUT", "900")
assert cc.cursor_timeout_seconds() == 900.0

67
tests/test_delta.py Normal file
View File

@@ -0,0 +1,67 @@
from __future__ import annotations
import json
import pytest
def test_build_movement_context_detects_new_skill(isolated_output, monkeypatch):
import daily.config as config
import daily.delta as delta
monkeypatch.setattr(config, "OUTPUT_DIR", isolated_output)
monkeypatch.setattr(delta, "OUTPUT_DIR", isolated_output)
prev_path = isolated_output / "2026-07-02.data.json"
prev_path.write_text(
json.dumps(
{
"data": {
"date": "2026-07-02",
"movement_baseline": {
"skills_trending": [{"id": "a/old", "title": "old"}],
"skills_hot": [],
"github_trending": [],
"github_emerging": [],
"github_topic": [],
},
}
},
ensure_ascii=False,
),
encoding="utf-8",
)
trending = [
{"id": "a/new", "title": "new-skill", "source": "a", "installs": 10, "link": "https://x", "description": ""},
{"id": "a/old", "title": "old", "source": "a", "installs": 9, "link": "https://y", "description": ""},
]
movement = delta.build_movement_context(
date_str="2026-07-03",
trending=trending,
hot=[],
github_trending=[],
github_emerging=[],
github_topic=[],
)
assert movement["baseline_date"] == "2026-07-02"
assert len(movement["skills_trending_moves"]) == 1
assert movement["skills_trending_moves"][0]["id"] == "a/new"
def test_find_previous_data_skips_missing_days(isolated_output, monkeypatch):
import daily.config as config
import daily.delta as delta
monkeypatch.setattr(config, "OUTPUT_DIR", isolated_output)
monkeypatch.setattr(delta, "OUTPUT_DIR", isolated_output)
(isolated_output / "2026-07-01.data.json").write_text(
json.dumps({"data": {"date": "2026-07-01", "skills_trending": []}}),
encoding="utf-8",
)
found = delta.find_previous_data("2026-07-03")
assert found is not None
assert found[0] == "2026-07-01"

View File

@@ -0,0 +1,40 @@
from __future__ import annotations
import pytest
def test_load_categories_from_yaml():
from daily.news.feeds_loader import load_categories
categories = load_categories("intl")
assert categories
assert any(cat.id == "official" for cat in categories)
assert categories[0].feeds
def test_load_categories_fallback_when_yaml_missing(monkeypatch, tmp_path):
import daily.config as config
import daily.news.feeds_loader as loader
missing = tmp_path / "missing" / "feeds.yaml"
monkeypatch.setattr(loader, "_FEEDS_FILE", missing)
loader._yaml_payload.cache_clear()
categories = loader.load_categories("intl")
assert categories
assert any(cat.id == "official" for cat in categories)
def test_content_filter_removes_matching_items(monkeypatch):
import daily.news.content_filter as cf
monkeypatch.setenv("DAILY_CONTENT_FILTER", "1")
cf.load_sensitive_words.cache_clear()
items = [
{"title": "正常 AI 新闻", "summary": "OpenAI 发布新模型"},
{"title": "违规推广", "summary": "六合彩内幕消息"},
]
kept, removed = cf.filter_news_items(items)
assert removed == 1
assert len(kept) == 1
assert kept[0]["title"].startswith("正常")

33
tests/test_llm_client.py Normal file
View File

@@ -0,0 +1,33 @@
from __future__ import annotations
import pytest
def test_resolve_llm_backend_prefers_cursor_in_agent_mode(monkeypatch):
import daily.llm_client as llm
monkeypatch.setenv("DAILY_REPORT_MODE", "agent")
monkeypatch.setenv("CURSOR_API_KEY", "cursor_test")
monkeypatch.setenv("DAILY_LLM_API_KEY", "openai_test")
monkeypatch.delenv("DAILY_LLM_PROVIDER", raising=False)
assert llm.resolve_llm_backend() == "cursor"
def test_resolve_llm_backend_openai_override(monkeypatch):
import daily.llm_client as llm
monkeypatch.setenv("DAILY_REPORT_MODE", "agent")
monkeypatch.setenv("DAILY_LLM_PROVIDER", "openai")
monkeypatch.setenv("CURSOR_API_KEY", "cursor_test")
monkeypatch.setenv("DAILY_LLM_API_KEY", "openai_test")
assert llm.resolve_llm_backend() == "openai"
def test_resolve_llm_backend_classic_defaults_openai(monkeypatch):
import daily.llm_client as llm
monkeypatch.setenv("DAILY_REPORT_MODE", "classic")
monkeypatch.setenv("CURSOR_API_KEY", "cursor_test")
monkeypatch.setenv("DAILY_LLM_API_KEY", "openai_test")
monkeypatch.delenv("DAILY_LLM_PROVIDER", raising=False)
assert llm.resolve_llm_backend() == "openai"

92
tests/test_news_rank.py Normal file
View File

@@ -0,0 +1,92 @@
from __future__ import annotations
from datetime import datetime, timezone
from daily.news.rank import (
apply_news_ranking,
fuzzy_dedupe_by_title,
jaccard_similarity,
score_news_item,
_title_tokens,
)
def test_jaccard_similarity_detects_duplicate_headlines():
left = _title_tokens("OpenAI launches GPT-5 with new reasoning")
right = _title_tokens("OpenAI launches GPT 5 with reasoning upgrade")
assert jaccard_similarity(left, right) >= 0.55
def test_fuzzy_dedupe_keeps_higher_score():
items = [
{"title": "OpenAI launches GPT-5", "link": "https://a.example/1", "score": 20},
{"title": "OpenAI launches GPT 5 today", "link": "https://b.example/2", "score": 45},
]
deduped = fuzzy_dedupe_by_title(items, threshold=0.55)
assert len(deduped) == 1
assert deduped[0]["link"] == "https://b.example/2"
def test_score_news_item_prefers_fresh_official_and_penalizes_yesterday():
now = datetime(2026, 7, 3, 12, 0, tzinfo=timezone.utc)
fresh = {
"title": "Claude update",
"link": "https://anthropic.com/news/claude",
"category_id": "official",
"published": "Thu, 03 Jul 2026 10:00:00 GMT",
}
stale_seen = {
"title": "Old story",
"link": "https://example.com/old",
"category_id": "media",
"published": "Mon, 30 Jun 2026 10:00:00 GMT",
}
fresh_score = score_news_item(fresh, yesterday_links=set(), now=now)
stale_score = score_news_item(
stale_seen,
yesterday_links={"https://example.com/old"},
now=now,
)
assert fresh_score > stale_score
def test_apply_news_ranking_sets_flat_scores(monkeypatch, tmp_path):
import daily.config as config
import daily.news.rank as rank
out = tmp_path / "output"
out.mkdir()
monkeypatch.setattr(config, "OUTPUT_DIR", out)
monkeypatch.setattr(rank, "load_yesterday_news_links", lambda _date: set())
news = {
"enabled": True,
"categories": [
{
"id": "official",
"name": "厂商官方",
"icon": "🏢",
"items": [
{
"title": "OpenAI ships new model",
"link": "https://openai.com/a",
"category_id": "official",
"published": "Thu, 03 Jul 2026 08:00:00 GMT",
},
{
"title": "OpenAI ships a new model today",
"link": "https://techcrunch.com/a",
"category_id": "media",
"published": "Thu, 03 Jul 2026 07:00:00 GMT",
},
],
}
],
"flat": [],
"stats": {},
}
ranked = apply_news_ranking(news, date_str="2026-07-03")
assert ranked["flat"]
assert all("score" in item for item in ranked["flat"])
assert len(ranked["flat"]) == 1

96
tests/test_rss_parse.py Normal file
View File

@@ -0,0 +1,96 @@
from __future__ import annotations
from pathlib import Path
import pytest
FIXTURES = Path(__file__).parent / "fixtures"
def test_parse_sample_rss_fixture():
from daily.news.feeds import NEWS_CATEGORIES
from daily.news.fetch import _parse_feed
category = NEWS_CATEGORIES[0]
feed_name = category.feeds[0].name
xml = (FIXTURES / "sample-rss.xml").read_text(encoding="utf-8")
items = _parse_feed(xml, feed_name, category)
assert len(items) == 1
assert items[0]["title"] == "Sample AI headline for smoke tests"
assert items[0]["link"] == "https://example.com/ai-news/1"
assert items[0]["source_name"] == feed_name
assert "minimal RSS item" in items[0]["summary"]
def test_rss_user_agent_avoids_bot_blocked_feeds():
from daily.news.fetch import RSS_USER_AGENT
assert "daily-robots" not in RSS_USER_AGENT
def test_fetch_one_retries_transient_errors(monkeypatch):
import httpx
from daily.news.feeds import NEWS_CATEGORIES
from daily.news import fetch as news_fetch
monkeypatch.setenv("DAILY_RSS_RETRY", "2")
monkeypatch.setattr(news_fetch.time, "sleep", lambda _: None)
category = NEWS_CATEGORIES[0]
feed = category.feeds[0]
calls = {"n": 0}
class FakeResponse:
def raise_for_status(self):
return None
@property
def text(self):
return (FIXTURES / "sample-rss.xml").read_text(encoding="utf-8")
class FakeClient:
headers = {"User-Agent": "test"}
def get(self, url, headers=None):
calls["n"] += 1
if calls["n"] < 2:
raise httpx.RemoteProtocolError("Server disconnected")
return FakeResponse()
entries, ok = news_fetch._fetch_one(FakeClient(), category, feed)
assert ok is True
assert len(entries) == 1
assert calls["n"] == 2
def test_fetch_ai_news_offline(monkeypatch):
from daily.news import fetch as news_fetch
sample = {
"title": "Sample AI headline for smoke tests",
"link": "https://example.com/ai-news/1",
"summary": "A minimal RSS item used by offline tests.",
"published": "Wed, 02 Jul 2026 08:00:00 GMT",
"source_name": "OpenAI",
"category_id": "official",
"category_name": "厂商官方",
"category_icon": "🏢",
}
monkeypatch.setattr(
news_fetch,
"_fetch_news",
lambda categories: {
"enabled": True,
"hours": 72,
"categories": [{"id": "official", "name": "厂商官方", "icon": "🏢", "items": [sample]}],
"flat": [sample],
"stats": {"feeds_total": 1, "feeds_ok": 1, "items_raw": 1, "feeds_failed": []},
},
)
payload = news_fetch.fetch_ai_news()
assert payload["enabled"] is True
assert payload["flat"][0]["title"].startswith("Sample AI")

67
tests/test_smoke.py Normal file
View File

@@ -0,0 +1,67 @@
from __future__ import annotations
import json
from pathlib import Path
import pytest
FIXTURES = Path(__file__).parent / "fixtures"
def _sample_github_repo() -> dict:
return {
"repo": "owner/sample",
"url": "https://github.com/owner/sample",
"description": "Sample repository for smoke tests.",
"language": "Python",
"stars_today_fmt": "120",
"total_stars_fmt": "1.2K",
"created_at": "2026-06-01",
}
def test_format_installs():
from shared.skills_data import format_installs
assert format_installs(999) == "999"
assert format_installs(18000) == "18.0K"
assert format_installs(2_500_000) == "2.5M"
def test_generate_report_offline(
skills_feed,
fixed_cst,
isolated_output,
offline_generate_env,
monkeypatch,
):
import daily.generate as generate
import daily.pipeline.collect as collect
monkeypatch.setattr(collect, "load_feed", lambda force=False: skills_feed)
monkeypatch.setattr(collect, "fetch_github_trending", lambda n: [_sample_github_repo()])
monkeypatch.setattr(collect, "fetch_emerging_repos", lambda n, exclude=None: [])
monkeypatch.setattr(collect, "fetch_topic_hot_repos", lambda n, exclude=None: ("llm", []))
monkeypatch.setattr(generate, "_now_cst", lambda: fixed_cst)
_, wecom_md, out_md, out_wecom = generate.generate_report()
assert out_md.exists()
assert out_wecom.exists()
assert "早报" in out_md.read_text(encoding="utf-8")
assert wecom_md.strip()
assert "📰" in wecom_md
data_path = isolated_output / "2026-07-03.data.json"
assert data_path.exists()
payload = json.loads(data_path.read_text(encoding="utf-8"))
assert payload["meta"]["report_mode"] == "classic"
assert payload["data"]["date"] == "2026-07-03"
assert len(payload["data"]["skills_trending"]) >= 1
assert len(payload["data"]["skills_hot"]) >= 1
def test_rss_fixture_is_well_formed():
xml = (FIXTURES / "sample-rss.xml").read_text(encoding="utf-8")
assert "Sample AI headline" in xml
assert "<item>" in xml

34
tests/test_wecom_split.py Normal file
View File

@@ -0,0 +1,34 @@
from __future__ import annotations
from daily.wecom_split import split_wecom_messages
def test_split_wecom_messages_keeps_short_text():
text = "📰 **早报 · 2026-07-03**\n\n短内容"
parts = split_wecom_messages(text, limit=4096)
assert parts == [text]
def test_split_wecom_messages_adds_part_footer():
section = "📰 **早报**\n\n" + ("正文行\n" * 200)
parts = split_wecom_messages(section, limit=600)
assert len(parts) > 1
assert all(len(part.encode("utf-8")) <= 600 for part in parts)
assert parts[0].endswith("1/" + str(len(parts)))
def test_split_sections_on_emoji_headers():
from daily.wecom_split import _split_sections
text = "\n\n".join(
[
"📰 **早报 · 2026-07-03**",
"💡 **今日速览**\n- item one\n- item two",
"🌍 **国际 AI 时讯**\n1. [Headline](https://example.com)",
]
)
sections = _split_sections(text)
assert len(sections) == 3
assert sections[0].startswith("📰")
assert sections[1].startswith("💡")
assert sections[2].startswith("🌍")

View File

@@ -1,14 +0,0 @@
import re
import certifi
import httpx
url = "https://skills.sh/vercel-labs/skills/find-skills"
r = httpx.get(url, headers={"User-Agent": "Mozilla/5.0"}, timeout=20, verify=certifi.where())
chunks = re.findall(r'self\.__next_f\.push\(\[1,"(.*?)"\]\)', r.text, re.DOTALL)
blob = "\n".join(chunks).encode("utf-8").decode("unicode_escape", errors="ignore")
for needle in ("description", "Helps users", "SKILL.md", "summary"):
print(needle, blob.count(needle))
idx = blob.find("Helps users")
if idx >= 0:
print(blob[idx : idx + 300])

Binary file not shown.