Compare commits

...

6 Commits

Author SHA1 Message Date
db270013eb fix: retry Agent Step2 once when first write attempt fails
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 16:40:05 +08:00
64179820d7 fix: retry transient RSS fetch failures with backoff
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 16:39:11 +08:00
f6fac7b642 fix: use browser User-Agent for RSS feeds blocked by InfoQ
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 16:38:48 +08:00
37931d2ef5 fix: raise Cursor SDK bridge timeout for daily LLM calls
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 16:38:21 +08:00
3936467838 删除 git actions 的相关配置 2026-07-03 15:30:41 +08:00
5742dc47ed feat: LLM 解耦 bot、RSS 外置与内容过滤
daily 自建 Cursor bridge;DAILY_LLM_PROVIDER 控制后端;config/feeds.yaml 与 sensitive_words 可配置。

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-03 15:24:58 +08:00
25 changed files with 927 additions and 362 deletions

View File

@@ -1,9 +1,6 @@
# 企微群机器人 webhook早报推送与 bot API 模式凭证不同) # 企微群机器人 webhook早报推送与 bot API 模式凭证不同)
WECOM_WEBHOOK_KEY=your-webhook-key WECOM_WEBHOOK_KEY=your-webhook-key
# GitHub Actions将上述 key 与下方可选项写入 repo Secrets / Variables
# 详见 README「部署GitHub Actions / Docker
# 早报内容 # 早报内容
DAILY_TRENDING_LIMIT=150 DAILY_TRENDING_LIMIT=150
DAILY_HOT_LIMIT=150 DAILY_HOT_LIMIT=150
@@ -62,8 +59,13 @@ DAILY_CN_AI_NEWS=1
# Agent 工作流(推荐:叙事化早报) # Agent 工作流(推荐:叙事化早报)
# DAILY_REPORT_MODE=agent # DAILY_REPORT_MODE=agent
# DAILY_LLM_PROVIDER=auto
# auto — agent 模式优先 Cursor否则 OpenAIclassic 优先 OpenAI
# openai — 强制 OpenAI 兼容 API
# cursor — 强制 Cursor SDK
# CURSOR_API_KEY=cursor_... # CURSOR_API_KEY=cursor_...
# CURSOR_MODEL=composer-2.5 # CURSOR_MODEL=composer-2.5
# CURSOR_TIMEOUT=600
# DAILY_CURSOR_CWD=. # DAILY_CURSOR_CWD=.
# 新增榜对比(较昨日 Top15供 Agent 导语/signals列表展示 Top N # 新增榜对比(较昨日 Top15供 Agent 导语/signals列表展示 Top N
@@ -83,6 +85,12 @@ DAILY_AI_NEWS_PER_CATEGORY=5
# DAILY_AGENT_NEWS_POOL=40 # DAILY_AGENT_NEWS_POOL=40
# DAILY_AGENT_CN_NEWS_POOL=30 # DAILY_AGENT_CN_NEWS_POOL=30
# RSS 源:编辑 config/feeds.yamlintl / cn缺失时回退内置默认
# 单源失败重试次数(默认 3含首次请求
# DAILY_RSS_RETRY=3
# 内容过滤config/sensitive_words.yaml + DAILY_CONTENT_FILTER=1
# DAILY_CONTENT_FILTER=0
# Reddit RSS403/429 时在 Reddit 偏好设置 → RSS feeds 复制 user / feed 参数) # Reddit RSS403/429 时在 Reddit 偏好设置 → RSS feeds 复制 user / feed 参数)
# REDDIT_RSS_USER=your_username # REDDIT_RSS_USER=your_username
# REDDIT_RSS_FEED=your_feed_token # REDDIT_RSS_FEED=your_feed_token

View File

@@ -1,64 +0,0 @@
name: daily
on:
schedule:
# 08:30 CST (UTC+8) = 00:30 UTC
- cron: "30 0 * * *"
workflow_dispatch:
inputs:
skip_push:
description: Skip WeCom push (generate only)
type: boolean
default: false
concurrency:
group: daily-report
cancel-in-progress: false
jobs:
report:
runs-on: ubuntu-latest
timeout-minutes: 30
env:
TZ: Asia/Shanghai
WECOM_WEBHOOK_KEY: ${{ secrets.WECOM_WEBHOOK_KEY }}
GITHUB_TOKEN: ${{ secrets.GH_PAT || github.token }}
CURSOR_API_KEY: ${{ secrets.CURSOR_API_KEY }}
DAILY_LLM_API_KEY: ${{ secrets.DAILY_LLM_API_KEY }}
DAILY_LLM_API_BASE: ${{ vars.DAILY_LLM_API_BASE }}
DAILY_LLM_MODEL: ${{ vars.DAILY_LLM_MODEL }}
DAILY_REPORT_MODE: ${{ vars.DAILY_REPORT_MODE }}
CURSOR_MODEL: ${{ vars.CURSOR_MODEL }}
DAILY_AI_NEWS: ${{ vars.DAILY_AI_NEWS }}
DAILY_CN_AI_NEWS: ${{ vars.DAILY_CN_AI_NEWS }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: pip
- name: Install dependencies
run: pip install -r requirements.txt
- name: Generate and push daily report
run: |
ARGS=()
if [ "${{ inputs.skip_push }}" = "true" ]; then
ARGS+=(--skip-push)
fi
chmod +x ./run-daily.sh
./run-daily.sh --force "${ARGS[@]}"
- name: Upload report artifacts
if: always()
uses: actions/upload-artifact@v4
with:
name: daily-report-${{ github.run_id }}
path: |
output/*.md
output/*.json
if-no-files-found: ignore
retention-days: 14

View File

@@ -1,20 +0,0 @@
name: test
on:
push:
branches: [main]
pull_request:
jobs:
pytest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: pip
- name: Install dependencies
run: pip install -r requirements-dev.txt
- name: Run tests
run: pytest -q

View File

@@ -16,7 +16,7 @@ RUN pip install --no-cache-dir -r requirements.txt
COPY daily/ daily/ COPY daily/ daily/
COPY shared/ shared/ COPY shared/ shared/
COPY skills/ skills/ COPY skills/ skills/
COPY bot/ bot/ COPY config/ config/
COPY run-daily.sh . COPY run-daily.sh .
RUN chmod +x run-daily.sh \ RUN chmod +x run-daily.sh \

View File

@@ -36,7 +36,7 @@
python -m daily push output\2026-07-03.wecom.md # 只推送 python -m daily push output\2026-07-03.wecom.md # 只推送
``` ```
Linux / macOS / CI 等价脚本: Linux / macOS 等价脚本:
```bash ```bash
chmod +x run-daily.sh chmod +x run-daily.sh
@@ -49,28 +49,7 @@ chmod +x run-daily.sh
--- ---
## 部署(GitHub Actions / Docker ## 部署Docker
### GitHub Actions零服务器定时推送
仓库已包含 [`.github/workflows/daily.yml`](.github/workflows/daily.yml),默认每天 **08:30北京时间** 生成并推送。
1. Fork 或启用本仓库的 Actions
2. **Settings → Secrets and variables → Actions** 添加 Secrets
| Secret | 必需 | 说明 |
|--------|------|------|
| `WECOM_WEBHOOK_KEY` | 是 | 企微群机器人 webhook key |
| `GH_PAT` | 否 | 个人 GitHub PAT提高 API 限额;不设则用 Actions 内置 token |
| `CURSOR_API_KEY` | 否 | Agent 模式Cursor SDK |
| `DAILY_LLM_API_KEY` | 否 | Agent / 编辑层OpenAI 兼容 API |
3. 可选 **Variables**(不设则用代码默认值):`DAILY_REPORT_MODE`、`DAILY_LLM_MODEL`、`DAILY_AI_NEWS` 等
4. **Actions → daily → Run workflow** 可手动触发;勾选 *Skip WeCom push* 则只生成不上报
运行产物可在 workflow 的 **Artifacts** 中下载(`.md` / `.json`,保留 14 天)。
### Docker
```bash ```bash
cp .env.example .env # 填入 WECOM_WEBHOOK_KEY 等 cp .env.example .env # 填入 WECOM_WEBHOOK_KEY 等
@@ -116,10 +95,11 @@ Python 抓取 → Step1 趋势分析 (.trends.json) → Step2 写企微稿 (.wec
## 自定义 RSS 源 ## 自定义 RSS 源
国际 RSS 列表:`daily/news/feeds.py` RSS 源配置在 **`config/feeds.yaml`**(国际 `intl` / 国内 `cn`),改 URL 或增删 feed 后无需改 Python。文件缺失或解析失败时会回退 `daily/news/feeds_defaults.py` 内置列表。
国内 RSS 列表:`daily/news/feeds_cn.py`
在对应文件的 `FEEDS` 列表中增删 URL 即可。常用开关(见 `.env.example` 可选 **`config/sensitive_words.yaml`** + `DAILY_CONTENT_FILTER=1`:标题/摘要命中敏感词则丢弃该条。
旧版硬编码路径(仍可读作参考):`daily/news/feeds_defaults.py`
```env ```env
DAILY_AI_NEWS=1 DAILY_AI_NEWS=1

166
config/feeds.yaml Normal file
View File

@@ -0,0 +1,166 @@
version: 1
intl:
categories:
- id: official
name: 厂商官方
icon: 🏢
feeds:
- name: Anthropic Claude 更新
url: https://docs.anthropic.com/en/release-notes/feed
- name: OpenAI
url: https://openai.com/news/rss.xml
- name: Google AI
url: https://blog.google/technology/ai/rss/
- name: DeepMind
url: https://deepmind.google/blog/rss.xml
- name: Meta Engineering
url: https://engineering.fb.com/feed/
- name: Microsoft Research
url: https://www.microsoft.com/en-us/research/feed/
- name: Microsoft Blog
url: https://blogs.microsoft.com/feed/
- name: Cohere
url: https://cohere.com/blog/rss.xml
- name: Cursor Changelog
url: https://cursor.com/changelog/rss.xml
- id: developer
name: Agent / LLM 开发者
icon: 🛠
feeds:
- name: LangChain
url: https://blog.langchain.dev/rss/
- name: Hugging Face
url: https://huggingface.co/blog/feed.xml
- name: Vercel Changelog
url: https://vercel.com/changelog/rss.xml
- name: GitHub Copilot
url: https://github.blog/changelog/label/copilot/feed/
- id: media
name: 综合科技媒体
icon: 📰
feeds:
- name: The Verge AI
url: https://www.theverge.com/rss/ai-artificial-intelligence/index.xml
- name: TechCrunch AI
url: https://techcrunch.com/category/artificial-intelligence/feed/
- name: Ars Technica AI
url: https://arstechnica.com/ai/feed/
- name: Wired AI
url: https://www.wired.com/feed/tag/ai/latest/rss
- name: MIT Tech Review
url: https://www.technologyreview.com/feed/
- name: VentureBeat AI
url: https://venturebeat.com/category/ai/feed/
- id: newsletter
name: Newsletter 日报
icon: ✉️
feeds:
- name: Ben's Bites
url: https://bensbites.substack.com/feed
- name: The Rundown AI
url: https://therundown.substack.com/feed
- name: Latent Space
url: https://www.latent.space/feed
- name: Simon Willison
url: https://simonwillison.net/atom/everything/
- name: Import AI
url: https://importai.substack.com/feed
- name: Last Week in AI
url: https://lastweekin.ai/feed
- name: The Neuron
url: https://www.theneuron.ai/feed
- id: research
name: 研究 / 论文
icon: 📚
feeds:
- name: arXiv cs.CL
url: https://arxiv.org/rss/cs.CL
- name: arXiv cs.AI
url: https://arxiv.org/rss/cs.AI
- name: arXiv cs.LG
url: https://arxiv.org/rss/cs.LG
- id: trending
name: 热点 / 趋势
icon: 🔥
feeds:
- name: Google News · AI
url: https://news.google.com/rss/search?q=artificial+intelligence+OR+LLM+OR+Claude+OR+GPT&hl=en-US&gl=US&ceid=US:en
- name: Google News · Technology
url: https://news.google.com/rss/headlines/section/topic/TECHNOLOGY?hl=en-US&gl=US&ceid=US:en
- name: Techmeme
url: https://www.techmeme.com/feed.xml
- name: HN · Front Page
url: https://hnrss.org/frontpage
- name: HN · 100+ Points
url: https://hnrss.org/newest?points=100
- name: Dev.to · AI
url: https://dev.to/feed/tag/ai
- name: Lobsters
url: https://lobste.rs/rss
- id: community
name: 社区讨论
icon: 💬
feeds:
- name: HN · AI/LLM/Agent
url: https://hnrss.org/newest?q=AI+OR+LLM+OR+Claude+OR+agent+OR+GPT+OR+Gemini
- name: Reddit · LLM/Claude/ML
url: https://old.reddit.com/r/LocalLLaMA+ClaudeAI+MachineLearning+OpenAI/.rss?limit=25
slow: true
cn:
title_keywords:
- 人工智能
- 大模型
- 智能体
- 多模态
- AIGC
- LLM
- GPT
- Claude
- Gemini
- ChatGPT
- OpenAI
- Anthropic
- Copilot
- Agent
- 'AI '
- ' AI'
- AI·
- AI业务
- AI模型
- AI助手
- AI工具
- AI编程
- AI 编程
- AI版
- AI Agent
- 推理模型
- 深度学习
- 机器学习
- Function Calling
categories:
- id: media
name: AI 专业媒体
icon: 📰
feeds:
- name: 量子位
url: https://www.qbitai.com/feed
- name: InfoQ 中文
url: https://www.infoq.cn/feed/AI
- id: tech
name: 综合科技
icon: 📱
feeds:
- name: 36氪
url: https://36kr.com/feed
ai_filter: true
- name: 雷锋网
url: https://www.leiphone.com/feed
- name: Google News · AI
url: https://news.google.com/rss/search?q=人工智能+OR+大模型+OR+Agent+OR+LLM&hl=zh-CN&gl=CN&ceid=CN:zh-Hans
- id: dev
name: 开发者社区
icon: 💻
feeds:
- name: 掘金
url: https://juejin.cn/rss
ai_filter: true

View File

@@ -0,0 +1,10 @@
# 标题或摘要命中任一词则丢弃(不区分大小写)
words:
- 赌博
- 六合彩
- 网赌
- 色情
- 裸聊
- 代孕
- 办证
- 刷单兼职

View File

@@ -125,10 +125,20 @@ def run_agent_workflow(
trends = analyze_trends(llm_input, date_str=date_str) trends = analyze_trends(llm_input, date_str=date_str)
if not trends: if not trends:
return None return None
return write_wecom_report( md = write_wecom_report(
llm_input, llm_input,
trends, trends,
date_str=date_str, date_str=date_str,
time_str=time_str, time_str=time_str,
updated=updated, updated=updated,
) )
if md is None:
logger.warning("Agent Step2 首次失败,重试一次")
md = write_wecom_report(
llm_input,
trends,
date_str=date_str,
time_str=time_str,
updated=updated,
)
return md

142
daily/cursor_bridge.py Normal file
View File

@@ -0,0 +1,142 @@
"""Windows 兼容的 Cursor SDK bridgedaily 包自用,不依赖 bot/)。"""
from __future__ import annotations
import codecs
import json
import logging
import os
import subprocess
import threading
import time
from pathlib import Path
from typing import Any, Mapping
from daily.config import ROOT, env
logger = logging.getLogger(__name__)
READY_LINE_PREFIX = "cursor-sdk-bridge ready "
_bridge_lock = threading.Lock()
_bridge_process: subprocess.Popen[bytes] | None = None
def cursor_cwd() -> str:
return env("DAILY_CURSOR_CWD") or env("CURSOR_CWD") or str(ROOT)
def _parse_discovery_line(line: str) -> Mapping[str, Any] | None:
if not line.startswith(READY_LINE_PREFIX):
return None
payload = line[len(READY_LINE_PREFIX) :].strip()
loaded = json.loads(payload)
if not isinstance(loaded, dict):
raise RuntimeError("Bridge discovery payload must be an object")
return loaded
def _read_discovery_polling(process: subprocess.Popen[bytes], timeout: float = 60) -> Mapping[str, Any]:
if process.stderr is None:
raise RuntimeError("Bridge stderr unavailable")
fd = process.stderr.fileno()
was_blocking = os.get_blocking(fd)
os.set_blocking(fd, False)
decoder = codecs.getincrementaldecoder("utf-8")(errors="replace")
pending = ""
stderr_lines: list[str] = []
deadline = time.monotonic() + timeout
try:
while time.monotonic() < deadline:
try:
chunk = os.read(fd, 8192)
except BlockingIOError:
chunk = b""
if chunk:
pending += decoder.decode(chunk)
while "\n" in pending:
line, pending = pending.split("\n", 1)
stderr_lines.append(line)
discovery = _parse_discovery_line(line)
if discovery is not None:
return discovery
else:
code = process.poll()
if code is not None:
pending += decoder.decode(b"", final=True)
if pending.strip():
stderr_lines.append(pending.strip())
joined = "\n".join(stderr_lines)[-2000:]
raise RuntimeError(
f"Bridge 启动失败 exit={code}: {joined or '无 stderr 输出'}"
)
time.sleep(0.05)
finally:
os.set_blocking(fd, was_blocking)
raise RuntimeError("等待 Cursor bridge 就绪超时")
def _auth_token_from_discovery(discovery: Mapping[str, Any]) -> str:
token = str(discovery.get("authToken") or "").strip()
if token:
return token
token_file = discovery.get("authTokenFile")
if token_file:
return Path(str(token_file)).read_text(encoding="utf-8").strip()
raise RuntimeError("Bridge discovery 缺少 auth token")
def warm_cursor_bridge(force: bool = False) -> None:
"""启动 cursor-sdk-bridge 并写入 CURSOR_SDK_BRIDGE_* 环境变量。"""
global _bridge_process
with _bridge_lock:
if (
not force
and _bridge_process is not None
and _bridge_process.poll() is None
and os.environ.get("CURSOR_SDK_BRIDGE_URL")
and os.environ.get("CURSOR_SDK_BRIDGE_TOKEN")
):
return
if _bridge_process is not None and _bridge_process.poll() is None:
_bridge_process.terminate()
try:
_bridge_process.wait(timeout=5)
except subprocess.TimeoutExpired:
_bridge_process.kill()
from cursor_sdk._vendor import resolve_bridge_path
cwd = cursor_cwd()
os.environ["CURSOR_CWD"] = cwd
argv = [resolve_bridge_path(), "--workspace", cwd]
logger.info("启动 Cursor bridge workspace=%s", cwd)
process = subprocess.Popen(
argv,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
)
try:
discovery = _read_discovery_polling(process)
except Exception:
process.kill()
process.wait(timeout=5)
raise
url = str(discovery.get("url") or "").strip()
if not url:
host = str(discovery.get("host") or "127.0.0.1")
port = discovery.get("port")
url = f"http://{host}:{port}"
token = _auth_token_from_discovery(discovery)
os.environ["CURSOR_SDK_BRIDGE_URL"] = url
os.environ["CURSOR_SDK_BRIDGE_TOKEN"] = token
_bridge_process = process
logger.info("Cursor bridge 就绪: %s", url)

68
daily/cursor_client.py Normal file
View File

@@ -0,0 +1,68 @@
"""Cursor SDK 调用daily 包专用)。"""
from __future__ import annotations
import os
from cursor_sdk import Agent, AgentOptions, Client, CursorAgentError, LocalAgentOptions
from daily.config import env, env_int
from daily.cursor_bridge import cursor_cwd, warm_cursor_bridge
_sdk_client: Client | None = None
_sdk_client_key: tuple[str, str, float] | None = None
def cursor_timeout_seconds() -> float:
"""Cursor bridge unary/stream 超时(秒);与 bot 共用 CURSOR_TIMEOUT。"""
return float(env_int("CURSOR_TIMEOUT", 600))
def _cursor_sdk_client() -> Client:
"""带自定义超时的 bridge ClientSDK 默认 unary 仅 60s复杂 Agent 任务易超时)。"""
global _sdk_client, _sdk_client_key
warm_cursor_bridge()
url = os.environ.get("CURSOR_SDK_BRIDGE_URL", "")
token = os.environ.get("CURSOR_SDK_BRIDGE_TOKEN", "")
timeout = cursor_timeout_seconds()
key = (url, token, timeout)
if _sdk_client is not None and _sdk_client_key == key:
return _sdk_client
if _sdk_client is not None:
_sdk_client.close()
_sdk_client = Client(
base_url=url,
auth_token=token,
unary_timeout=timeout,
stream_timeout=timeout,
allow_api_key_env_fallback=False,
)
_sdk_client_key = key
return _sdk_client
def cursor_chat(system: str, user: str) -> str:
api_key = (env("CURSOR_API_KEY") or "").strip()
if not api_key:
return ""
client = _cursor_sdk_client()
cwd = cursor_cwd()
model = env("CURSOR_MODEL") or "composer-2.5"
prompt = f"{system}\n\n{user}"
try:
result = Agent.prompt(
prompt,
AgentOptions(
api_key=api_key,
model=model,
local=LocalAgentOptions(cwd=cwd),
),
client=client,
)
except CursorAgentError as exc:
raise RuntimeError(f"LLM 调用失败:{exc.message}") from exc
if result.status == "error":
raise RuntimeError(f"LLM 调用失败:{result.result or '未知错误'}")
return (result.result or "").strip()

View File

@@ -3,17 +3,14 @@
from __future__ import annotations from __future__ import annotations
import json import json
import logging
import os
import re import re
from typing import Any from typing import Any
import certifi import certifi
import httpx import httpx
from daily.config import ROOT, env, env_int from daily.config import env, env_int
from daily.cursor_client import cursor_chat
logger = logging.getLogger(__name__)
_JSON_BLOCK = re.compile(r"```(?:json)?\s*([\s\S]*?)```", re.IGNORECASE) _JSON_BLOCK = re.compile(r"```(?:json)?\s*([\s\S]*?)```", re.IGNORECASE)
@@ -44,6 +41,47 @@ def extract_json_object(text: str) -> dict[str, Any]:
return {} return {}
def _has_openai_configured() -> bool:
return bool(env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY"))
def _has_cursor_configured() -> bool:
return bool(env("CURSOR_API_KEY"))
def llm_provider() -> str:
raw = (env("DAILY_LLM_PROVIDER") or "auto").strip().lower()
if raw in {"openai", "cursor"}:
return raw
return "auto"
def resolve_llm_backend() -> str:
"""返回 openai | cursor | 空字符串。"""
provider = llm_provider()
has_openai = _has_openai_configured()
has_cursor = _has_cursor_configured()
if provider == "openai":
if has_openai:
return "openai"
return "cursor" if has_cursor else ""
if provider == "cursor":
if has_cursor:
return "cursor"
return "openai" if has_openai else ""
agent_mode = (env("DAILY_REPORT_MODE") or "").strip().lower() == "agent"
if agent_mode and has_cursor:
return "cursor"
if has_openai:
return "openai"
if has_cursor:
return "cursor"
return ""
def _openai_chat(system: str, user: str) -> str: def _openai_chat(system: str, user: str) -> str:
api_key = (env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY") or "").strip() api_key = (env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY") or "").strip()
if not api_key: if not api_key:
@@ -70,53 +108,14 @@ def _openai_chat(system: str, user: str) -> str:
return str(data["choices"][0]["message"]["content"] or "").strip() return str(data["choices"][0]["message"]["content"] or "").strip()
def _cursor_chat(system: str, user: str) -> str:
api_key = (env("CURSOR_API_KEY") or "").strip()
if not api_key:
return ""
import sys
from daily.config import ROOT
from cursor_sdk import Agent, AgentOptions, CursorAgentError, LocalAgentOptions
_bot = str(ROOT / "bot")
if _bot not in sys.path:
sys.path.insert(0, _bot)
try:
from bridge_manager import warm_cursor_bridge
except ImportError:
warm_cursor_bridge = lambda: None # noqa: E731
cwd = env("DAILY_CURSOR_CWD") or str(ROOT)
# bridge_manager 读 bot env_config 的 CURSOR_CWD早报侧须先对齐工作目录
os.environ["CURSOR_CWD"] = cwd
warm_cursor_bridge()
model = env("CURSOR_MODEL") or "composer-2.5"
prompt = f"{system}\n\n{user}"
try:
result = Agent.prompt(
prompt,
AgentOptions(
api_key=api_key,
model=model,
local=LocalAgentOptions(cwd=cwd),
),
)
except CursorAgentError as exc:
raise RuntimeError(f"LLM 调用失败:{exc.message}") from exc
if result.status == "error":
raise RuntimeError(f"LLM 调用失败:{result.result or '未知错误'}")
return (result.result or "").strip()
def llm_chat(system: str, user: str) -> str: def llm_chat(system: str, user: str) -> str:
"""优先 OpenAI 兼容 API否则 Cursor SDK。""" backend = resolve_llm_backend()
if env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY"): if backend == "openai":
return _openai_chat(system, user) return _openai_chat(system, user)
if env("CURSOR_API_KEY"): if backend == "cursor":
return _cursor_chat(system, user) return cursor_chat(system, user)
return "" return ""
def has_llm_configured() -> bool: def has_llm_configured() -> bool:
return bool(env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY") or env("CURSOR_API_KEY")) return bool(resolve_llm_backend())

View File

@@ -0,0 +1,77 @@
"""敏感词内容过滤。"""
from __future__ import annotations
import logging
from functools import lru_cache
from pathlib import Path
from typing import Any
from daily.config import ROOT, env
logger = logging.getLogger(__name__)
_WORDS_FILE = ROOT / "config" / "sensitive_words.yaml"
def content_filter_enabled() -> bool:
raw = (env("DAILY_CONTENT_FILTER") or "0").strip().lower()
return raw in {"1", "true", "yes", "on"}
@lru_cache(maxsize=1)
def load_sensitive_words() -> tuple[str, ...]:
if not _WORDS_FILE.exists():
return ()
try:
import yaml
except ImportError:
logger.warning("未安装 PyYAML无法读取 %s", _WORDS_FILE)
return ()
try:
data = yaml.safe_load(_WORDS_FILE.read_text(encoding="utf-8"))
except OSError as exc:
logger.warning("读取敏感词配置失败: %s", exc)
return ()
except Exception as exc:
logger.warning("解析 sensitive_words.yaml 失败: %s", exc)
return ()
if not isinstance(data, dict):
return ()
words = data.get("words") or data.get("sensitive_words") or []
if not isinstance(words, list):
return ()
cleaned = tuple(str(word).strip() for word in words if str(word).strip())
return cleaned
def matches_sensitive_text(text: str, words: tuple[str, ...]) -> str | None:
haystack = (text or "").lower()
if not haystack:
return None
for word in words:
needle = word.lower()
if needle and needle in haystack:
return word
return None
def filter_news_items(items: list[dict[str, Any]]) -> tuple[list[dict[str, Any]], int]:
if not content_filter_enabled():
return items, 0
words = load_sensitive_words()
if not words:
return items, 0
kept: list[dict[str, Any]] = []
removed = 0
for item in items:
text = f"{item.get('title', '')} {item.get('summary', '')}"
hit = matches_sensitive_text(text, words)
if hit:
removed += 1
continue
kept.append(item)
if removed:
logger.info("内容过滤移除 %d 条(敏感词)", removed)
return kept, removed

View File

@@ -1,125 +1,10 @@
"""国际 AI 时讯 RSS 源定义(按类别分组)。""" """国际 AI 时讯 RSS 源(优先 config/feeds.yaml)。"""
from __future__ import annotations from __future__ import annotations
from dataclasses import dataclass from daily.news.feeds_loader import load_categories
from daily.news.feeds_types import NewsCategory, NewsFeed
NEWS_CATEGORIES: tuple[NewsCategory, ...] = load_categories("intl")
@dataclass(frozen=True) __all__ = ["NewsCategory", "NewsFeed", "NEWS_CATEGORIES"]
class NewsFeed:
name: str
url: str
slow: bool = False # 限速源(如 Reddit串行抓取
ai_filter: bool = False # 综合源仅保留标题命中 AI 关键词的条目
@dataclass(frozen=True)
class NewsCategory:
id: str
name: str
icon: str
feeds: tuple[NewsFeed, ...]
NEWS_CATEGORIES: tuple[NewsCategory, ...] = (
NewsCategory(
id="official",
name="厂商官方",
icon="🏢",
feeds=(
NewsFeed("Anthropic Claude 更新", "https://docs.anthropic.com/en/release-notes/feed"),
NewsFeed("OpenAI", "https://openai.com/news/rss.xml"),
NewsFeed("Google AI", "https://blog.google/technology/ai/rss/"),
NewsFeed("DeepMind", "https://deepmind.google/blog/rss.xml"),
NewsFeed("Meta Engineering", "https://engineering.fb.com/feed/"),
NewsFeed("Microsoft Research", "https://www.microsoft.com/en-us/research/feed/"),
NewsFeed("Microsoft Blog", "https://blogs.microsoft.com/feed/"),
NewsFeed("Cohere", "https://cohere.com/blog/rss.xml"),
NewsFeed("Cursor Changelog", "https://cursor.com/changelog/rss.xml"),
),
),
NewsCategory(
id="developer",
name="Agent / LLM 开发者",
icon="🛠",
feeds=(
NewsFeed("LangChain", "https://blog.langchain.dev/rss/"),
NewsFeed("Hugging Face", "https://huggingface.co/blog/feed.xml"),
NewsFeed("Vercel Changelog", "https://vercel.com/changelog/rss.xml"),
NewsFeed("GitHub Copilot", "https://github.blog/changelog/label/copilot/feed/"),
),
),
NewsCategory(
id="media",
name="综合科技媒体",
icon="📰",
feeds=(
NewsFeed("The Verge AI", "https://www.theverge.com/rss/ai-artificial-intelligence/index.xml"),
NewsFeed("TechCrunch AI", "https://techcrunch.com/category/artificial-intelligence/feed/"),
NewsFeed("Ars Technica AI", "https://arstechnica.com/ai/feed/"),
NewsFeed("Wired AI", "https://www.wired.com/feed/tag/ai/latest/rss"),
NewsFeed("MIT Tech Review", "https://www.technologyreview.com/feed/"),
NewsFeed("VentureBeat AI", "https://venturebeat.com/category/ai/feed/"),
),
),
NewsCategory(
id="newsletter",
name="Newsletter 日报",
icon="✉️",
feeds=(
NewsFeed("Ben's Bites", "https://bensbites.substack.com/feed"),
NewsFeed("The Rundown AI", "https://therundown.substack.com/feed"),
NewsFeed("Latent Space", "https://www.latent.space/feed"),
NewsFeed("Simon Willison", "https://simonwillison.net/atom/everything/"),
NewsFeed("Import AI", "https://importai.substack.com/feed"),
NewsFeed("Last Week in AI", "https://lastweekin.ai/feed"),
NewsFeed("The Neuron", "https://www.theneuron.ai/feed"),
),
),
NewsCategory(
id="research",
name="研究 / 论文",
icon="📚",
feeds=(
NewsFeed("arXiv cs.CL", "https://arxiv.org/rss/cs.CL"),
NewsFeed("arXiv cs.AI", "https://arxiv.org/rss/cs.AI"),
NewsFeed("arXiv cs.LG", "https://arxiv.org/rss/cs.LG"),
),
),
NewsCategory(
id="trending",
name="热点 / 趋势",
icon="🔥",
feeds=(
NewsFeed(
"Google News · AI",
"https://news.google.com/rss/search?q=artificial+intelligence+OR+LLM+OR+Claude+OR+GPT&hl=en-US&gl=US&ceid=US:en",
),
NewsFeed(
"Google News · Technology",
"https://news.google.com/rss/headlines/section/topic/TECHNOLOGY?hl=en-US&gl=US&ceid=US:en",
),
NewsFeed("Techmeme", "https://www.techmeme.com/feed.xml"),
NewsFeed("HN · Front Page", "https://hnrss.org/frontpage"),
NewsFeed("HN · 100+ Points", "https://hnrss.org/newest?points=100"),
NewsFeed("Dev.to · AI", "https://dev.to/feed/tag/ai"),
NewsFeed("Lobsters", "https://lobste.rs/rss"),
),
),
NewsCategory(
id="community",
name="社区讨论",
icon="💬",
feeds=(
NewsFeed(
"HN · AI/LLM/Agent",
"https://hnrss.org/newest?q=AI+OR+LLM+OR+Claude+OR+agent+OR+GPT+OR+Gemini",
),
NewsFeed(
"Reddit · LLM/Claude/ML",
"https://old.reddit.com/r/LocalLLaMA+ClaudeAI+MachineLearning+OpenAI/.rss?limit=25",
slow=True,
),
),
),
)

View File

@@ -1,71 +1,11 @@
"""国内 AI 时讯 RSS 源定义(按类别分组)。""" """国内 AI 时讯 RSS 源(优先 config/feeds.yaml)。"""
from __future__ import annotations from __future__ import annotations
from daily.news.feeds import NewsCategory, NewsFeed from daily.news.feeds_loader import load_categories, load_cn_title_keywords
from daily.news.feeds_types import NewsCategory, NewsFeed
# 综合源 ai_filter=True 时,仅保留标题命中以下词之一的条目 CN_AI_TITLE_KEYWORDS: tuple[str, ...] = load_cn_title_keywords()
CN_AI_TITLE_KEYWORDS: tuple[str, ...] = ( CN_NEWS_CATEGORIES: tuple[NewsCategory, ...] = load_categories("cn")
"人工智能",
"大模型",
"智能体",
"多模态",
"AIGC",
"LLM",
"GPT",
"Claude",
"Gemini",
"ChatGPT",
"OpenAI",
"Anthropic",
"Copilot",
"Agent",
"AI ",
" AI",
"AI·",
"AI业务",
"AI模型",
"AI助手",
"AI工具",
"AI编程",
"AI 编程",
"AI版",
"AI Agent",
"推理模型",
"深度学习",
"机器学习",
"Function Calling",
)
CN_NEWS_CATEGORIES: tuple[NewsCategory, ...] = ( __all__ = ["CN_AI_TITLE_KEYWORDS", "CN_NEWS_CATEGORIES", "NewsCategory", "NewsFeed"]
NewsCategory(
id="media",
name="AI 专业媒体",
icon="📰",
feeds=(
NewsFeed("量子位", "https://www.qbitai.com/feed"),
NewsFeed("InfoQ 中文", "https://www.infoq.cn/feed/AI"),
),
),
NewsCategory(
id="tech",
name="综合科技",
icon="📱",
feeds=(
NewsFeed("36氪", "https://36kr.com/feed", ai_filter=True),
NewsFeed("雷锋网", "https://www.leiphone.com/feed"),
NewsFeed(
"Google News · AI",
"https://news.google.com/rss/search?q=人工智能+OR+大模型+OR+Agent+OR+LLM&hl=zh-CN&gl=CN&ceid=CN:zh-Hans",
),
),
),
NewsCategory(
id="dev",
name="开发者社区",
icon="💻",
feeds=(
NewsFeed("掘金", "https://juejin.cn/rss", ai_filter=True),
),
),
)

View File

@@ -0,0 +1,11 @@
"""Built-in RSS defaults when config/feeds.yaml is missing or invalid."""
from __future__ import annotations
from daily.news.feeds_types import NewsCategory, NewsFeed
CN_AI_TITLE_KEYWORDS: tuple[str, ...] = ('人工智能', '大模型', '智能体', '多模态', 'AIGC', 'LLM', 'GPT', 'Claude', 'Gemini', 'ChatGPT', 'OpenAI', 'Anthropic', 'Copilot', 'Agent', 'AI ', ' AI', 'AI·', 'AI业务', 'AI模型', 'AI助手', 'AI工具', 'AI编程', 'AI 编程', 'AI版', 'AI Agent', '推理模型', '深度学习', '机器学习', 'Function Calling')
CN_NEWS_CATEGORIES: tuple[NewsCategory, ...] = (NewsCategory(id='media', name='AI 专业媒体', icon='📰', feeds=(NewsFeed(name='量子位', url='https://www.qbitai.com/feed', slow=False, ai_filter=False), NewsFeed(name='InfoQ 中文', url='https://www.infoq.cn/feed/AI', slow=False, ai_filter=False))), NewsCategory(id='tech', name='综合科技', icon='📱', feeds=(NewsFeed(name='36氪', url='https://36kr.com/feed', slow=False, ai_filter=True), NewsFeed(name='雷锋网', url='https://www.leiphone.com/feed', slow=False, ai_filter=False), NewsFeed(name='Google News · AI', url='https://news.google.com/rss/search?q=人工智能+OR+大模型+OR+Agent+OR+LLM&hl=zh-CN&gl=CN&ceid=CN:zh-Hans', slow=False, ai_filter=False))), NewsCategory(id='dev', name='开发者社区', icon='💻', feeds=(NewsFeed(name='掘金', url='https://juejin.cn/rss', slow=False, ai_filter=True),)))
NEWS_CATEGORIES: tuple[NewsCategory, ...] = (NewsCategory(id='official', name='厂商官方', icon='🏢', feeds=(NewsFeed(name='Anthropic Claude 更新', url='https://docs.anthropic.com/en/release-notes/feed', slow=False, ai_filter=False), NewsFeed(name='OpenAI', url='https://openai.com/news/rss.xml', slow=False, ai_filter=False), NewsFeed(name='Google AI', url='https://blog.google/technology/ai/rss/', slow=False, ai_filter=False), NewsFeed(name='DeepMind', url='https://deepmind.google/blog/rss.xml', slow=False, ai_filter=False), NewsFeed(name='Meta Engineering', url='https://engineering.fb.com/feed/', slow=False, ai_filter=False), NewsFeed(name='Microsoft Research', url='https://www.microsoft.com/en-us/research/feed/', slow=False, ai_filter=False), NewsFeed(name='Microsoft Blog', url='https://blogs.microsoft.com/feed/', slow=False, ai_filter=False), NewsFeed(name='Cohere', url='https://cohere.com/blog/rss.xml', slow=False, ai_filter=False), NewsFeed(name='Cursor Changelog', url='https://cursor.com/changelog/rss.xml', slow=False, ai_filter=False))), NewsCategory(id='developer', name='Agent / LLM 开发者', icon='🛠', feeds=(NewsFeed(name='LangChain', url='https://blog.langchain.dev/rss/', slow=False, ai_filter=False), NewsFeed(name='Hugging Face', url='https://huggingface.co/blog/feed.xml', slow=False, ai_filter=False), NewsFeed(name='Vercel Changelog', url='https://vercel.com/changelog/rss.xml', slow=False, ai_filter=False), NewsFeed(name='GitHub Copilot', url='https://github.blog/changelog/label/copilot/feed/', slow=False, ai_filter=False))), NewsCategory(id='media', name='综合科技媒体', icon='📰', feeds=(NewsFeed(name='The Verge AI', url='https://www.theverge.com/rss/ai-artificial-intelligence/index.xml', slow=False, ai_filter=False), NewsFeed(name='TechCrunch AI', url='https://techcrunch.com/category/artificial-intelligence/feed/', slow=False, ai_filter=False), NewsFeed(name='Ars Technica AI', url='https://arstechnica.com/ai/feed/', slow=False, ai_filter=False), NewsFeed(name='Wired AI', url='https://www.wired.com/feed/tag/ai/latest/rss', slow=False, ai_filter=False), NewsFeed(name='MIT Tech Review', url='https://www.technologyreview.com/feed/', slow=False, ai_filter=False), NewsFeed(name='VentureBeat AI', url='https://venturebeat.com/category/ai/feed/', slow=False, ai_filter=False))), NewsCategory(id='newsletter', name='Newsletter 日报', icon='✉️', feeds=(NewsFeed(name="Ben's Bites", url='https://bensbites.substack.com/feed', slow=False, ai_filter=False), NewsFeed(name='The Rundown AI', url='https://therundown.substack.com/feed', slow=False, ai_filter=False), NewsFeed(name='Latent Space', url='https://www.latent.space/feed', slow=False, ai_filter=False), NewsFeed(name='Simon Willison', url='https://simonwillison.net/atom/everything/', slow=False, ai_filter=False), NewsFeed(name='Import AI', url='https://importai.substack.com/feed', slow=False, ai_filter=False), NewsFeed(name='Last Week in AI', url='https://lastweekin.ai/feed', slow=False, ai_filter=False), NewsFeed(name='The Neuron', url='https://www.theneuron.ai/feed', slow=False, ai_filter=False))), NewsCategory(id='research', name='研究 / 论文', icon='📚', feeds=(NewsFeed(name='arXiv cs.CL', url='https://arxiv.org/rss/cs.CL', slow=False, ai_filter=False), NewsFeed(name='arXiv cs.AI', url='https://arxiv.org/rss/cs.AI', slow=False, ai_filter=False), NewsFeed(name='arXiv cs.LG', url='https://arxiv.org/rss/cs.LG', slow=False, ai_filter=False))), NewsCategory(id='trending', name='热点 / 趋势', icon='🔥', feeds=(NewsFeed(name='Google News · AI', url='https://news.google.com/rss/search?q=artificial+intelligence+OR+LLM+OR+Claude+OR+GPT&hl=en-US&gl=US&ceid=US:en', slow=False, ai_filter=False), NewsFeed(name='Google News · Technology', url='https://news.google.com/rss/headlines/section/topic/TECHNOLOGY?hl=en-US&gl=US&ceid=US:en', slow=False, ai_filter=False), NewsFeed(name='Techmeme', url='https://www.techmeme.com/feed.xml', slow=False, ai_filter=False), NewsFeed(name='HN · Front Page', url='https://hnrss.org/frontpage', slow=False, ai_filter=False), NewsFeed(name='HN · 100+ Points', url='https://hnrss.org/newest?points=100', slow=False, ai_filter=False), NewsFeed(name='Dev.to · AI', url='https://dev.to/feed/tag/ai', slow=False, ai_filter=False), NewsFeed(name='Lobsters', url='https://lobste.rs/rss', slow=False, ai_filter=False))), NewsCategory(id='community', name='社区讨论', icon='💬', feeds=(NewsFeed(name='HN · AI/LLM/Agent', url='https://hnrss.org/newest?q=AI+OR+LLM+OR+Claude+OR+agent+OR+GPT+OR+Gemini', slow=False, ai_filter=False), NewsFeed(name='Reddit · LLM/Claude/ML', url='https://old.reddit.com/r/LocalLLaMA+ClaudeAI+MachineLearning+OpenAI/.rss?limit=25', slow=True, ai_filter=False))))

113
daily/news/feeds_loader.py Normal file
View File

@@ -0,0 +1,113 @@
"""从 config/feeds.yaml 加载 RSS 源(失败时回退内置默认)。"""
from __future__ import annotations
import logging
from functools import lru_cache
from pathlib import Path
from typing import Any
from daily.config import ROOT
from daily.news.feeds_types import NewsCategory, NewsFeed
logger = logging.getLogger(__name__)
_FEEDS_FILE = ROOT / "config" / "feeds.yaml"
def _parse_feed(raw: dict[str, Any]) -> NewsFeed:
return NewsFeed(
name=str(raw.get("name") or "").strip(),
url=str(raw.get("url") or "").strip(),
slow=bool(raw.get("slow")),
ai_filter=bool(raw.get("ai_filter")),
)
def _parse_category(raw: dict[str, Any]) -> NewsCategory | None:
cat_id = str(raw.get("id") or "").strip()
if not cat_id:
return None
feeds_raw = raw.get("feeds") or []
feeds: list[NewsFeed] = []
for item in feeds_raw:
if not isinstance(item, dict):
continue
feed = _parse_feed(item)
if feed.name and feed.url:
feeds.append(feed)
if not feeds:
return None
return NewsCategory(
id=cat_id,
name=str(raw.get("name") or cat_id),
icon=str(raw.get("icon") or "📰"),
feeds=tuple(feeds),
)
def _parse_categories(items: Any) -> tuple[NewsCategory, ...]:
if not isinstance(items, list):
return ()
categories: list[NewsCategory] = []
for raw in items:
if not isinstance(raw, dict):
continue
cat = _parse_category(raw)
if cat:
categories.append(cat)
return tuple(categories)
def _load_yaml() -> dict[str, Any] | None:
if not _FEEDS_FILE.exists():
return None
try:
import yaml
except ImportError:
logger.warning("未安装 PyYAML无法读取 %s", _FEEDS_FILE)
return None
try:
data = yaml.safe_load(_FEEDS_FILE.read_text(encoding="utf-8"))
except OSError as exc:
logger.warning("读取 feeds 配置失败: %s", exc)
return None
except Exception as exc:
logger.warning("解析 feeds.yaml 失败: %s", exc)
return None
return data if isinstance(data, dict) else None
@lru_cache(maxsize=1)
def _yaml_payload() -> dict[str, Any] | None:
return _load_yaml()
def load_categories(region: str) -> tuple[NewsCategory, ...]:
data = _yaml_payload()
if data:
block = data.get(region) or {}
categories = _parse_categories(block.get("categories"))
if categories:
return categories
logger.warning("feeds.yaml 中 %s.categories 为空,使用内置默认", region)
from daily.news import feeds_defaults as defaults
if region == "cn":
return defaults.CN_NEWS_CATEGORIES
return defaults.NEWS_CATEGORIES
def load_cn_title_keywords() -> tuple[str, ...]:
data = _yaml_payload()
if data:
block = data.get("cn") or {}
keywords = block.get("title_keywords")
if isinstance(keywords, list):
cleaned = tuple(str(x).strip() for x in keywords if str(x).strip())
if cleaned:
return cleaned
from daily.news.feeds_defaults import CN_AI_TITLE_KEYWORDS
return CN_AI_TITLE_KEYWORDS

21
daily/news/feeds_types.py Normal file
View File

@@ -0,0 +1,21 @@
"""RSS 源数据结构。"""
from __future__ import annotations
from dataclasses import dataclass
@dataclass(frozen=True)
class NewsFeed:
name: str
url: str
slow: bool = False
ai_filter: bool = False
@dataclass(frozen=True)
class NewsCategory:
id: str
name: str
icon: str
feeds: tuple[NewsFeed, ...]

View File

@@ -17,17 +17,19 @@ import certifi
import httpx import httpx
from daily.config import env, env_int, news_summary_limit from daily.config import env, env_int, news_summary_limit
from daily.news.content_filter import filter_news_items
from daily.news.feeds import NEWS_CATEGORIES, NewsCategory, NewsFeed from daily.news.feeds import NEWS_CATEGORIES, NewsCategory, NewsFeed
from daily.news.feeds_cn import CN_AI_TITLE_KEYWORDS, CN_NEWS_CATEGORIES from daily.news.feeds_cn import CN_AI_TITLE_KEYWORDS, CN_NEWS_CATEGORIES
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
USER_AGENT = "Mozilla/5.0 (compatible; daily-robots/1.0; +https://skills.sh)" # 部分站点(如 InfoQ会拦截含 bot 标识的 UARSS 抓取统一用浏览器 UA
BROWSER_USER_AGENT = ( RSS_USER_AGENT = (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) " "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) " "AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/131.0.0.0 Safari/537.36" "Chrome/131.0.0.0 Safari/537.36"
) )
BROWSER_USER_AGENT = RSS_USER_AGENT
STRIP_HTML = re.compile(r"<[^>]+>") STRIP_HTML = re.compile(r"<[^>]+>")
WS = re.compile(r"\s+") WS = re.compile(r"\s+")
@@ -287,17 +289,22 @@ def _fetch_one(
category: NewsCategory, category: NewsCategory,
feed: NewsFeed, feed: NewsFeed,
) -> tuple[list[dict[str, Any]], bool]: ) -> tuple[list[dict[str, Any]], bool]:
max_attempts = max(1, env_int("DAILY_RSS_RETRY", 3))
last_exc: Exception | None = None last_exc: Exception | None = None
for url in _reddit_fetch_urls(feed.url): for url in _reddit_fetch_urls(feed.url):
try: for attempt in range(max_attempts):
headers = _request_headers(dict(client.headers), url) try:
resp = client.get(url, headers=headers) headers = _request_headers(dict(client.headers), url)
resp.raise_for_status() resp = client.get(url, headers=headers)
entries = _parse_feed(resp.text, feed.name, category) resp.raise_for_status()
return _filter_ai_entries(entries, ai_filter=feed.ai_filter), True entries = _parse_feed(resp.text, feed.name, category)
except Exception as exc: return _filter_ai_entries(entries, ai_filter=feed.ai_filter), True
last_exc = exc except Exception as exc:
continue last_exc = exc
if attempt + 1 < max_attempts:
time.sleep(min(2.0 * (attempt + 1), 5.0))
continue
break
logger.warning("RSS fetch failed [%s] %s: %s", feed.name, feed.url, last_exc) logger.warning("RSS fetch failed [%s] %s: %s", feed.name, feed.url, last_exc)
return [], False return [], False
@@ -337,7 +344,7 @@ def _fetch_news(categories: tuple[NewsCategory, ...]) -> dict[str, Any]:
per_category = _per_category_limit() per_category = _per_category_limit()
cutoff = _now_utc() - timedelta(hours=hours) cutoff = _now_utc() - timedelta(hours=hours)
headers = {"User-Agent": USER_AGENT, "Accept": "application/rss+xml, application/atom+xml, application/xml, text/xml, */*"} headers = {"User-Agent": RSS_USER_AGENT, "Accept": "application/rss+xml, application/atom+xml, application/xml, text/xml, */*"}
tasks: list[tuple[NewsCategory, NewsFeed]] = [] tasks: list[tuple[NewsCategory, NewsFeed]] = []
for category in categories: for category in categories:
for feed in category.feeds: for feed in category.feeds:
@@ -349,6 +356,7 @@ def _fetch_news(categories: tuple[NewsCategory, ...]) -> dict[str, Any]:
"feeds_ok": 0, "feeds_ok": 0,
"items_raw": 0, "items_raw": 0,
"feeds_failed": [], "feeds_failed": [],
"content_filtered": 0,
} }
with httpx.Client(timeout=15.0, verify=certifi.where(), follow_redirects=True, headers=headers) as client: with httpx.Client(timeout=15.0, verify=certifi.where(), follow_redirects=True, headers=headers) as client:
@@ -394,6 +402,9 @@ def _fetch_news(categories: tuple[NewsCategory, ...]) -> dict[str, Any]:
for category in categories: for category in categories:
items = raw_by_category[category.id] items = raw_by_category[category.id]
items = [i for i in items if _within_window(i, cutoff)] items = [i for i in items if _within_window(i, cutoff)]
items, filtered_count = filter_news_items(items)
if filtered_count:
stats["content_filtered"] = stats.get("content_filtered", 0) + filtered_count
items.sort(key=_sort_key, reverse=True) items.sort(key=_sort_key, reverse=True)
items = _dedupe_items(items)[:per_category] items = _dedupe_items(items)[:per_category]
for item in items: for item in items:

View File

@@ -1,5 +1,6 @@
python-dotenv>=1.0.0 python-dotenv>=1.0.0
httpx>=0.27.0 httpx>=0.27.0
certifi>=2024.0.0 certifi>=2024.0.0
PyYAML>=6.0.0
# 英文描述转中文(使用 CURSOR_API_KEY 时需安装) # 英文描述转中文(使用 CURSOR_API_KEY 时需安装)
cursor-sdk>=0.1.0 cursor-sdk>=0.1.0

View File

@@ -0,0 +1,47 @@
from pathlib import Path
import yaml
from daily.news.feeds_types import NewsCategory, NewsFeed
data = yaml.safe_load(Path("config/feeds.yaml").read_text(encoding="utf-8"))
def parse_feed(raw: dict) -> NewsFeed:
return NewsFeed(
name=raw["name"],
url=raw["url"],
slow=bool(raw.get("slow")),
ai_filter=bool(raw.get("ai_filter")),
)
def parse_cat(raw: dict) -> NewsCategory:
return NewsCategory(
id=raw["id"],
name=raw["name"],
icon=raw["icon"],
feeds=tuple(parse_feed(item) for item in raw["feeds"]),
)
intl = tuple(parse_cat(item) for item in data["intl"]["categories"])
cn = tuple(parse_cat(item) for item in data["cn"]["categories"])
keywords = tuple(data["cn"]["title_keywords"])
lines = [
'"""Built-in RSS defaults when config/feeds.yaml is missing or invalid."""',
"",
"from __future__ import annotations",
"",
"from daily.news.feeds_types import NewsCategory, NewsFeed",
"",
f"CN_AI_TITLE_KEYWORDS: tuple[str, ...] = {keywords!r}",
"",
f"CN_NEWS_CATEGORIES: tuple[NewsCategory, ...] = {cn!r}",
"",
f"NEWS_CATEGORIES: tuple[NewsCategory, ...] = {intl!r}",
"",
]
Path("daily/news/feeds_defaults.py").write_text("\n".join(lines), encoding="utf-8")
print("written")

View File

@@ -53,3 +53,33 @@ def test_write_wecom_report_extracts_markdown_block(monkeypatch):
assert md is not None assert md is not None
assert md.startswith("📰") assert md.startswith("📰")
assert "正文" in md assert "正文" in md
def test_run_agent_workflow_retries_step2(monkeypatch):
import daily.agent_workflow as agent
calls = {"n": 0}
def fake_llm(system, user):
if '"trends"' in user:
calls["n"] += 1
if calls["n"] == 1:
raise RuntimeError("Bridge request timed out")
return "📰 **早报 · 2026-07-03**\n\n重试成功"
return json.dumps(
{"headline": "h", "opening": "o", "themes": [], "top_picks": [], "signals": []},
ensure_ascii=False,
)
monkeypatch.setattr(agent, "llm_chat", fake_llm)
md = agent.run_agent_workflow(
{"date": "2026-07-03"},
date_str="2026-07-03",
time_str="09:30 (UTC+8)",
updated="2026-07-02",
)
assert md is not None
assert "重试成功" in md
assert calls["n"] == 2

View File

@@ -0,0 +1,15 @@
from __future__ import annotations
def test_cursor_timeout_seconds_default(monkeypatch):
import daily.cursor_client as cc
monkeypatch.delenv("CURSOR_TIMEOUT", raising=False)
assert cc.cursor_timeout_seconds() == 600.0
def test_cursor_timeout_seconds_from_env(monkeypatch):
import daily.cursor_client as cc
monkeypatch.setenv("CURSOR_TIMEOUT", "900")
assert cc.cursor_timeout_seconds() == 900.0

View File

@@ -0,0 +1,40 @@
from __future__ import annotations
import pytest
def test_load_categories_from_yaml():
from daily.news.feeds_loader import load_categories
categories = load_categories("intl")
assert categories
assert any(cat.id == "official" for cat in categories)
assert categories[0].feeds
def test_load_categories_fallback_when_yaml_missing(monkeypatch, tmp_path):
import daily.config as config
import daily.news.feeds_loader as loader
missing = tmp_path / "missing" / "feeds.yaml"
monkeypatch.setattr(loader, "_FEEDS_FILE", missing)
loader._yaml_payload.cache_clear()
categories = loader.load_categories("intl")
assert categories
assert any(cat.id == "official" for cat in categories)
def test_content_filter_removes_matching_items(monkeypatch):
import daily.news.content_filter as cf
monkeypatch.setenv("DAILY_CONTENT_FILTER", "1")
cf.load_sensitive_words.cache_clear()
items = [
{"title": "正常 AI 新闻", "summary": "OpenAI 发布新模型"},
{"title": "违规推广", "summary": "六合彩内幕消息"},
]
kept, removed = cf.filter_news_items(items)
assert removed == 1
assert len(kept) == 1
assert kept[0]["title"].startswith("正常")

33
tests/test_llm_client.py Normal file
View File

@@ -0,0 +1,33 @@
from __future__ import annotations
import pytest
def test_resolve_llm_backend_prefers_cursor_in_agent_mode(monkeypatch):
import daily.llm_client as llm
monkeypatch.setenv("DAILY_REPORT_MODE", "agent")
monkeypatch.setenv("CURSOR_API_KEY", "cursor_test")
monkeypatch.setenv("DAILY_LLM_API_KEY", "openai_test")
monkeypatch.delenv("DAILY_LLM_PROVIDER", raising=False)
assert llm.resolve_llm_backend() == "cursor"
def test_resolve_llm_backend_openai_override(monkeypatch):
import daily.llm_client as llm
monkeypatch.setenv("DAILY_REPORT_MODE", "agent")
monkeypatch.setenv("DAILY_LLM_PROVIDER", "openai")
monkeypatch.setenv("CURSOR_API_KEY", "cursor_test")
monkeypatch.setenv("DAILY_LLM_API_KEY", "openai_test")
assert llm.resolve_llm_backend() == "openai"
def test_resolve_llm_backend_classic_defaults_openai(monkeypatch):
import daily.llm_client as llm
monkeypatch.setenv("DAILY_REPORT_MODE", "classic")
monkeypatch.setenv("CURSOR_API_KEY", "cursor_test")
monkeypatch.setenv("DAILY_LLM_API_KEY", "openai_test")
monkeypatch.delenv("DAILY_LLM_PROVIDER", raising=False)
assert llm.resolve_llm_backend() == "openai"

View File

@@ -23,6 +23,48 @@ def test_parse_sample_rss_fixture():
assert "minimal RSS item" in items[0]["summary"] assert "minimal RSS item" in items[0]["summary"]
def test_rss_user_agent_avoids_bot_blocked_feeds():
from daily.news.fetch import RSS_USER_AGENT
assert "daily-robots" not in RSS_USER_AGENT
def test_fetch_one_retries_transient_errors(monkeypatch):
import httpx
from daily.news.feeds import NEWS_CATEGORIES
from daily.news import fetch as news_fetch
monkeypatch.setenv("DAILY_RSS_RETRY", "2")
monkeypatch.setattr(news_fetch.time, "sleep", lambda _: None)
category = NEWS_CATEGORIES[0]
feed = category.feeds[0]
calls = {"n": 0}
class FakeResponse:
def raise_for_status(self):
return None
@property
def text(self):
return (FIXTURES / "sample-rss.xml").read_text(encoding="utf-8")
class FakeClient:
headers = {"User-Agent": "test"}
def get(self, url, headers=None):
calls["n"] += 1
if calls["n"] < 2:
raise httpx.RemoteProtocolError("Server disconnected")
return FakeResponse()
entries, ok = news_fetch._fetch_one(FakeClient(), category, feed)
assert ok is True
assert len(entries) == 1
assert calls["n"] == 2
def test_fetch_ai_news_offline(monkeypatch): def test_fetch_ai_news_offline(monkeypatch):
from daily.news import fetch as news_fetch from daily.news import fetch as news_fetch