项目初始化

This commit is contained in:
2026-07-02 11:31:16 +08:00
commit eef4c76e3f
64 changed files with 13369 additions and 0 deletions

View File

@@ -0,0 +1,98 @@
---
name: brand-voice
description: Build a source-derived writing style profile from real posts, essays, launch notes, docs, or site copy, then reuse that profile across content, outreach, and social workflows. Use when the user wants voice consistency without generic AI writing tropes.
metadata:
origin: ECC
---
# Brand Voice
Build a durable voice profile from real source material, then use that profile everywhere instead of re-deriving style from scratch or defaulting to generic AI copy.
## When to Activate
- the user wants content or outreach in a specific voice
- writing for X, LinkedIn, email, launch posts, threads, or product updates
- adapting a known author's tone across channels
- the existing content lane needs a reusable style system instead of one-off mimicry
## Source Priority
Use the strongest real source set available, in this order:
1. recent original X posts and threads
2. articles, essays, memos, launch notes, or newsletters
3. real outbound emails or DMs that worked
4. product docs, changelogs, README framing, and site copy
Do not use generic platform exemplars as source material.
## Collection Workflow
1. Gather 5 to 20 representative samples when available.
2. Prefer recent material over old material unless the user says the older writing is more canonical.
3. Separate "public launch voice" from "private working voice" if the source set clearly splits.
4. If live X access is available, use `x-api` to pull recent original posts before drafting.
5. If site copy matters, include the current ECC landing page and repo/plugin framing.
## What to Extract
- rhythm and sentence length
- compression vs explanation
- capitalization norms
- parenthetical use
- question frequency and purpose
- how sharply claims are made
- how often numbers, mechanisms, or receipts show up
- how transitions work
- what the author never does
## Output Contract
Produce a reusable `VOICE PROFILE` block that downstream skills can consume directly. Use the schema in [references/voice-profile-schema.md](references/voice-profile-schema.md).
Keep the profile structured and short enough to reuse in session context. The point is not literary criticism. The point is operational reuse.
## Affaan / ECC Defaults
If the user wants Affaan / ECC voice and live sources are thin, start here unless newer source material overrides it:
- direct, compressed, concrete
- specifics, mechanisms, receipts, and numbers beat adjectives
- parentheticals are for qualification, narrowing, or over-clarification
- capitalization is conventional unless there is a real reason to break it
- questions are rare and should not be used as bait
- tone can be sharp, blunt, skeptical, or dry
- transitions should feel earned, not smoothed over
## Hard Bans
Delete and rewrite any of these:
- fake curiosity hooks
- "not X, just Y"
- "no fluff"
- forced lowercase
- LinkedIn thought-leader cadence
- bait questions
- "Excited to share"
- generic founder-journey filler
- corny parentheticals
## Persistence Rules
- Reuse the latest confirmed `VOICE PROFILE` across related tasks in the same session.
- If the user asks for a durable artifact, save the profile in the requested workspace location or memory surface.
- Do not create repo-tracked files that store personal voice fingerprints unless the user explicitly asks for that.
## Downstream Use
Use this skill before or inside:
- `content-engine`
- `crosspost`
- `lead-intelligence`
- article or launch writing
- cold or warm outbound across X, LinkedIn, and email
If another skill already has a partial voice capture section, this skill is the canonical source of truth.

View File

@@ -0,0 +1,55 @@
# Voice Profile Schema
Use this exact structure when building a reusable voice profile:
```text
VOICE PROFILE
=============
Author:
Goal:
Confidence:
Source Set
- source 1
- source 2
- source 3
Rhythm
- short note on sentence length, pacing, and fragmentation
Compression
- how dense or explanatory the writing is
Capitalization
- conventional, mixed, or situational
Parentheticals
- how they are used and how they are not used
Question Use
- rare, frequent, rhetorical, direct, or mostly absent
Claim Style
- how claims are framed, supported, and sharpened
Preferred Moves
- concrete moves the author does use
Banned Moves
- specific patterns the author does not use
CTA Rules
- how, when, or whether to close with asks
Channel Notes
- X:
- LinkedIn:
- Email:
```
Guidelines:
- Keep the profile concrete and source-backed.
- Use short bullets, not essay paragraphs.
- Every banned move should be observable in the source set or explicitly requested by the user.
- If the source set conflicts, call out the split instead of averaging it into mush.

View File

@@ -0,0 +1,751 @@
---
name: python-patterns
description: Pythonic idioms, PEP 8 standards, type hints, and best practices for building robust, efficient, and maintainable Python applications.
metadata:
origin: ECC
---
# Python Development Patterns
Idiomatic Python patterns and best practices for building robust, efficient, and maintainable applications.
## When to Activate
- Writing new Python code
- Reviewing Python code
- Refactoring existing Python code
- Designing Python packages/modules
## Core Principles
### 1. Readability Counts
Python prioritizes readability. Code should be obvious and easy to understand.
```python
# Good: Clear and readable
def get_active_users(users: list[User]) -> list[User]:
"""Return only active users from the provided list."""
return [user for user in users if user.is_active]
# Bad: Clever but confusing
def get_active_users(u):
return [x for x in u if x.a]
```
### 2. Explicit is Better Than Implicit
Avoid magic; be clear about what your code does.
```python
# Good: Explicit configuration
import logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
# Bad: Hidden side effects
import some_module
some_module.setup() # What does this do?
```
### 3. EAFP - Easier to Ask Forgiveness Than Permission
Python prefers exception handling over checking conditions.
```python
# Good: EAFP style
def get_value(dictionary: dict, key: str, default_value: Any = None) -> Any:
try:
return dictionary[key]
except KeyError:
return default_value
# Bad: LBYL (Look Before You Leap) style
def get_value(dictionary: dict, key: str, default_value: Any = None) -> Any:
if key in dictionary:
return dictionary[key]
else:
return default_value
```
## Type Hints
### Basic Type Annotations
```python
from typing import Optional, List, Dict, Any
def process_user(
user_id: str,
data: Dict[str, Any],
active: bool = True
) -> Optional[User]:
"""Process a user and return the updated User or None."""
if not active:
return None
return User(user_id, data)
```
### Modern Type Hints (Python 3.9+)
```python
# Python 3.9+ - Use built-in types
def process_items(items: list[str]) -> dict[str, int]:
return {item: len(item) for item in items}
# Python 3.8 and earlier - Use typing module
from typing import List, Dict
def process_items(items: List[str]) -> Dict[str, int]:
return {item: len(item) for item in items}
```
### Type Aliases and TypeVar
```python
from typing import TypeVar, Union
# Type alias for complex types
JSON = Union[dict[str, Any], list[Any], str, int, float, bool, None]
def parse_json(data: str) -> JSON:
return json.loads(data)
# Generic types
T = TypeVar('T')
def first(items: list[T]) -> T | None:
"""Return the first item or None if list is empty."""
return items[0] if items else None
```
### Protocol-Based Duck Typing
```python
from typing import Protocol
class Renderable(Protocol):
def render(self) -> str:
"""Render the object to a string."""
def render_all(items: list[Renderable]) -> str:
"""Render all items that implement the Renderable protocol."""
return "\n".join(item.render() for item in items)
```
## Error Handling Patterns
### Specific Exception Handling
```python
# Good: Catch specific exceptions
def load_config(path: str) -> Config:
try:
with open(path) as f:
return Config.from_json(f.read())
except FileNotFoundError as e:
raise ConfigError(f"Config file not found: {path}") from e
except json.JSONDecodeError as e:
raise ConfigError(f"Invalid JSON in config: {path}") from e
# Bad: Bare except
def load_config(path: str) -> Config:
try:
with open(path) as f:
return Config.from_json(f.read())
except:
return None # Silent failure!
```
### Exception Chaining
```python
def process_data(data: str) -> Result:
try:
parsed = json.loads(data)
except json.JSONDecodeError as e:
# Chain exceptions to preserve the traceback
raise ValueError(f"Failed to parse data: {data}") from e
```
### Custom Exception Hierarchy
```python
class AppError(Exception):
"""Base exception for all application errors."""
pass
class ValidationError(AppError):
"""Raised when input validation fails."""
pass
class NotFoundError(AppError):
"""Raised when a requested resource is not found."""
pass
# Usage
def get_user(user_id: str) -> User:
user = db.find_user(user_id)
if not user:
raise NotFoundError(f"User not found: {user_id}")
return user
```
## Context Managers
### Resource Management
```python
# Good: Using context managers
def process_file(path: str) -> str:
with open(path, 'r') as f:
return f.read()
# Bad: Manual resource management
def process_file(path: str) -> str:
f = open(path, 'r')
try:
return f.read()
finally:
f.close()
```
### Custom Context Managers
```python
from contextlib import contextmanager
@contextmanager
def timer(name: str):
"""Context manager to time a block of code."""
start = time.perf_counter()
yield
elapsed = time.perf_counter() - start
print(f"{name} took {elapsed:.4f} seconds")
# Usage
with timer("data processing"):
process_large_dataset()
```
### Context Manager Classes
```python
class DatabaseTransaction:
def __init__(self, connection):
self.connection = connection
def __enter__(self):
self.connection.begin_transaction()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
if exc_type is None:
self.connection.commit()
else:
self.connection.rollback()
return False # Don't suppress exceptions
# Usage
with DatabaseTransaction(conn):
user = conn.create_user(user_data)
conn.create_profile(user.id, profile_data)
```
## Comprehensions and Generators
### List Comprehensions
```python
# Good: List comprehension for simple transformations
names = [user.name for user in users if user.is_active]
# Bad: Manual loop
names = []
for user in users:
if user.is_active:
names.append(user.name)
# Complex comprehensions should be expanded
# Bad: Too complex
result = [x * 2 for x in items if x > 0 if x % 2 == 0]
# Good: Use a generator function
def filter_and_transform(items: Iterable[int]) -> list[int]:
result = []
for x in items:
if x > 0 and x % 2 == 0:
result.append(x * 2)
return result
```
### Generator Expressions
```python
# Good: Generator for lazy evaluation
total = sum(x * x for x in range(1_000_000))
# Bad: Creates large intermediate list
total = sum([x * x for x in range(1_000_000)])
```
### Generator Functions
```python
def read_large_file(path: str) -> Iterator[str]:
"""Read a large file line by line."""
with open(path) as f:
for line in f:
yield line.strip()
# Usage
for line in read_large_file("huge.txt"):
process(line)
```
## Data Classes and Named Tuples
### Data Classes
```python
from dataclasses import dataclass, field
from datetime import datetime
@dataclass
class User:
"""User entity with automatic __init__, __repr__, and __eq__."""
id: str
name: str
email: str
created_at: datetime = field(default_factory=datetime.now)
is_active: bool = True
# Usage
user = User(
id="123",
name="Alice",
email="alice@example.com"
)
```
### Data Classes with Validation
```python
@dataclass
class User:
email: str
age: int
def __post_init__(self):
# Validate email format
if "@" not in self.email:
raise ValueError(f"Invalid email: {self.email}")
# Validate age range
if self.age < 0 or self.age > 150:
raise ValueError(f"Invalid age: {self.age}")
```
### Named Tuples
```python
from typing import NamedTuple
class Point(NamedTuple):
"""Immutable 2D point."""
x: float
y: float
def distance(self, other: 'Point') -> float:
return ((self.x - other.x) ** 2 + (self.y - other.y) ** 2) ** 0.5
# Usage
p1 = Point(0, 0)
p2 = Point(3, 4)
print(p1.distance(p2)) # 5.0
```
## Decorators
### Function Decorators
```python
import functools
import time
def timer(func: Callable) -> Callable:
"""Decorator to time function execution."""
@functools.wraps(func)
def wrapper(*args, **kwargs):
start = time.perf_counter()
result = func(*args, **kwargs)
elapsed = time.perf_counter() - start
print(f"{func.__name__} took {elapsed:.4f}s")
return result
return wrapper
@timer
def slow_function():
time.sleep(1)
# slow_function() prints: slow_function took 1.0012s
```
### Parameterized Decorators
```python
def repeat(times: int):
"""Decorator to repeat a function multiple times."""
def decorator(func: Callable) -> Callable:
@functools.wraps(func)
def wrapper(*args, **kwargs):
results = []
for _ in range(times):
results.append(func(*args, **kwargs))
return results
return wrapper
return decorator
@repeat(times=3)
def greet(name: str) -> str:
return f"Hello, {name}!"
# greet("Alice") returns ["Hello, Alice!", "Hello, Alice!", "Hello, Alice!"]
```
### Class-Based Decorators
```python
class CountCalls:
"""Decorator that counts how many times a function is called."""
def __init__(self, func: Callable):
functools.update_wrapper(self, func)
self.func = func
self.count = 0
def __call__(self, *args, **kwargs):
self.count += 1
print(f"{self.func.__name__} has been called {self.count} times")
return self.func(*args, **kwargs)
@CountCalls
def process():
pass
# Each call to process() prints the call count
```
## Concurrency Patterns
### Threading for I/O-Bound Tasks
```python
import concurrent.futures
import threading
def fetch_url(url: str) -> str:
"""Fetch a URL (I/O-bound operation)."""
import urllib.request
with urllib.request.urlopen(url) as response:
return response.read().decode()
def fetch_all_urls(urls: list[str]) -> dict[str, str]:
"""Fetch multiple URLs concurrently using threads."""
with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
future_to_url = {executor.submit(fetch_url, url): url for url in urls}
results = {}
for future in concurrent.futures.as_completed(future_to_url):
url = future_to_url[future]
try:
results[url] = future.result()
except Exception as e:
results[url] = f"Error: {e}"
return results
```
### Multiprocessing for CPU-Bound Tasks
```python
def process_data(data: list[int]) -> int:
"""CPU-intensive computation."""
return sum(x ** 2 for x in data)
def process_all(datasets: list[list[int]]) -> list[int]:
"""Process multiple datasets using multiple processes."""
with concurrent.futures.ProcessPoolExecutor() as executor:
results = list(executor.map(process_data, datasets))
return results
```
### Async/Await for Concurrent I/O
```python
import asyncio
async def fetch_async(url: str) -> str:
"""Fetch a URL asynchronously."""
import aiohttp
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
return await response.text()
async def fetch_all(urls: list[str]) -> dict[str, str]:
"""Fetch multiple URLs concurrently."""
tasks = [fetch_async(url) for url in urls]
results = await asyncio.gather(*tasks, return_exceptions=True)
return dict(zip(urls, results))
```
## Package Organization
### Standard Project Layout
```
myproject/
├── src/
│ └── mypackage/
│ ├── __init__.py
│ ├── main.py
│ ├── api/
│ │ ├── __init__.py
│ │ └── routes.py
│ ├── models/
│ │ ├── __init__.py
│ │ └── user.py
│ └── utils/
│ ├── __init__.py
│ └── helpers.py
├── tests/
│ ├── __init__.py
│ ├── conftest.py
│ ├── test_api.py
│ └── test_models.py
├── pyproject.toml
├── README.md
└── .gitignore
```
### Import Conventions
```python
# Good: Import order - stdlib, third-party, local
import os
import sys
from pathlib import Path
import requests
from fastapi import FastAPI
from mypackage.models import User
from mypackage.utils import format_name
# Good: Use isort for automatic import sorting
# pip install isort
```
### __init__.py for Package Exports
```python
# mypackage/__init__.py
"""mypackage - A sample Python package."""
__version__ = "1.0.0"
# Export main classes/functions at package level
from mypackage.models import User, Post
from mypackage.utils import format_name
__all__ = ["User", "Post", "format_name"]
```
## Memory and Performance
### Using __slots__ for Memory Efficiency
```python
# Bad: Regular class uses __dict__ (more memory)
class Point:
def __init__(self, x: float, y: float):
self.x = x
self.y = y
# Good: __slots__ reduces memory usage
class Point:
__slots__ = ['x', 'y']
def __init__(self, x: float, y: float):
self.x = x
self.y = y
```
### Generator for Large Data
```python
# Bad: Returns full list in memory
def read_lines(path: str) -> list[str]:
with open(path) as f:
return [line.strip() for line in f]
# Good: Yields lines one at a time
def read_lines(path: str) -> Iterator[str]:
with open(path) as f:
for line in f:
yield line.strip()
```
### Avoid String Concatenation in Loops
```python
# Bad: O(n²) due to string immutability
result = ""
for item in items:
result += str(item)
# Good: O(n) using join
result = "".join(str(item) for item in items)
# Good: Using StringIO for building
from io import StringIO
buffer = StringIO()
for item in items:
buffer.write(str(item))
result = buffer.getvalue()
```
## Python Tooling Integration
### Essential Commands
```bash
# Code formatting
black .
isort .
# Linting
ruff check .
pylint mypackage/
# Type checking
mypy .
# Testing
pytest --cov=mypackage --cov-report=html
# Security scanning
bandit -r .
# Dependency management
pip-audit
safety check
```
### pyproject.toml Configuration
```toml
[project]
name = "mypackage"
version = "1.0.0"
requires-python = ">=3.9"
dependencies = [
"requests>=2.31.0",
"pydantic>=2.0.0",
]
[project.optional-dependencies]
dev = [
"pytest>=7.4.0",
"pytest-cov>=4.1.0",
"black>=23.0.0",
"ruff>=0.1.0",
"mypy>=1.5.0",
]
[tool.black]
line-length = 88
target-version = ['py39']
[tool.ruff]
line-length = 88
select = ["E", "F", "I", "N", "W"]
[tool.mypy]
python_version = "3.9"
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "--cov=mypackage --cov-report=term-missing"
```
## Quick Reference: Python Idioms
| Idiom | Description |
|-------|-------------|
| EAFP | Easier to Ask Forgiveness than Permission |
| Context managers | Use `with` for resource management |
| List comprehensions | For simple transformations |
| Generators | For lazy evaluation and large datasets |
| Type hints | Annotate function signatures |
| Dataclasses | For data containers with auto-generated methods |
| `__slots__` | For memory optimization |
| f-strings | For string formatting (Python 3.6+) |
| `pathlib.Path` | For path operations (Python 3.4+) |
| `enumerate` | For index-element pairs in loops |
## Anti-Patterns to Avoid
```python
# Bad: Mutable default arguments
def append_to(item, items=[]):
items.append(item)
return items
# Good: Use None and create new list
def append_to(item, items=None):
if items is None:
items = []
items.append(item)
return items
# Bad: Checking type with type()
if type(obj) == list:
process(obj)
# Good: Use isinstance
if isinstance(obj, list):
process(obj)
# Bad: Comparing to None with ==
if value == None:
process()
# Good: Use is
if value is None:
process()
# Bad: from module import *
from os.path import *
# Good: Explicit imports
from os.path import join, exists
# Bad: Bare except
try:
risky_operation()
except:
pass
# Good: Specific exception
try:
risky_operation()
except SpecificError as e:
logger.error(f"Operation failed: {e}")
```
__Remember__: Python code should be readable, explicit, and follow the principle of least surprise. When in doubt, prioritize clarity over cleverness.

View File

@@ -0,0 +1,817 @@
---
name: python-testing
description: Python testing strategies using pytest, TDD methodology, fixtures, mocking, parametrization, and coverage requirements.
metadata:
origin: ECC
---
# Python Testing Patterns
Comprehensive testing strategies for Python applications using pytest, TDD methodology, and best practices.
## When to Activate
- Writing new Python code (follow TDD: red, green, refactor)
- Designing test suites for Python projects
- Reviewing Python test coverage
- Setting up testing infrastructure
## Core Testing Philosophy
### Test-Driven Development (TDD)
Always follow the TDD cycle:
1. **RED**: Write a failing test for the desired behavior
2. **GREEN**: Write minimal code to make the test pass
3. **REFACTOR**: Improve code while keeping tests green
```python
# Step 1: Write failing test (RED)
def test_add_numbers():
result = add(2, 3)
assert result == 5
# Step 2: Write minimal implementation (GREEN)
def add(a, b):
return a + b
# Step 3: Refactor if needed (REFACTOR)
```
### Coverage Requirements
- **Target**: 80%+ code coverage
- **Critical paths**: 100% coverage required
- Use `pytest --cov` to measure coverage
```bash
pytest --cov=mypackage --cov-report=term-missing --cov-report=html
```
## pytest Fundamentals
### Basic Test Structure
```python
import pytest
def test_addition():
"""Test basic addition."""
assert 2 + 2 == 4
def test_string_uppercase():
"""Test string uppercasing."""
text = "hello"
assert text.upper() == "HELLO"
def test_list_append():
"""Test list append."""
items = [1, 2, 3]
items.append(4)
assert 4 in items
assert len(items) == 4
```
### Assertions
```python
# Equality
assert result == expected
# Inequality
assert result != unexpected
# Truthiness
assert result # Truthy
assert not result # Falsy
assert result is True # Exactly True
assert result is False # Exactly False
assert result is None # Exactly None
# Membership
assert item in collection
assert item not in collection
# Comparisons
assert result > 0
assert 0 <= result <= 100
# Type checking
assert isinstance(result, str)
# Exception testing (preferred approach)
with pytest.raises(ValueError):
raise ValueError("error message")
# Check exception message
with pytest.raises(ValueError, match="invalid input"):
raise ValueError("invalid input provided")
# Check exception attributes
with pytest.raises(ValueError) as exc_info:
raise ValueError("error message")
assert str(exc_info.value) == "error message"
```
## Fixtures
### Basic Fixture Usage
```python
import pytest
@pytest.fixture
def sample_data():
"""Fixture providing sample data."""
return {"name": "Alice", "age": 30}
def test_sample_data(sample_data):
"""Test using the fixture."""
assert sample_data["name"] == "Alice"
assert sample_data["age"] == 30
```
### Fixture with Setup/Teardown
```python
@pytest.fixture
def database():
"""Fixture with setup and teardown."""
# Setup
db = Database(":memory:")
db.create_tables()
db.insert_test_data()
yield db # Provide to test
# Teardown
db.close()
def test_database_query(database):
"""Test database operations."""
result = database.query("SELECT * FROM users")
assert len(result) > 0
```
### Fixture Scopes
```python
# Function scope (default) - runs for each test
@pytest.fixture
def temp_file():
with open("temp.txt", "w") as f:
yield f
os.remove("temp.txt")
# Module scope - runs once per module
@pytest.fixture(scope="module")
def module_db():
db = Database(":memory:")
db.create_tables()
yield db
db.close()
# Session scope - runs once per test session
@pytest.fixture(scope="session")
def shared_resource():
resource = ExpensiveResource()
yield resource
resource.cleanup()
```
### Fixture with Parameters
```python
@pytest.fixture(params=[1, 2, 3])
def number(request):
"""Parameterized fixture."""
return request.param
def test_numbers(number):
"""Test runs 3 times, once for each parameter."""
assert number > 0
```
### Using Multiple Fixtures
```python
@pytest.fixture
def user():
return User(id=1, name="Alice")
@pytest.fixture
def admin():
return User(id=2, name="Admin", role="admin")
def test_user_admin_interaction(user, admin):
"""Test using multiple fixtures."""
assert admin.can_manage(user)
```
### Autouse Fixtures
```python
@pytest.fixture(autouse=True)
def reset_config():
"""Automatically runs before every test."""
Config.reset()
yield
Config.cleanup()
def test_without_fixture_call():
# reset_config runs automatically
assert Config.get_setting("debug") is False
```
### Conftest.py for Shared Fixtures
```python
# tests/conftest.py
import pytest
@pytest.fixture
def client():
"""Shared fixture for all tests."""
app = create_app(testing=True)
with app.test_client() as client:
yield client
@pytest.fixture
def auth_headers(client):
"""Generate auth headers for API testing."""
response = client.post("/api/login", json={
"username": "test",
"password": "test"
})
token = response.json["token"]
return {"Authorization": f"Bearer {token}"}
```
## Parametrization
### Basic Parametrization
```python
@pytest.mark.parametrize("input,expected", [
("hello", "HELLO"),
("world", "WORLD"),
("PyThOn", "PYTHON"),
])
def test_uppercase(input, expected):
"""Test runs 3 times with different inputs."""
assert input.upper() == expected
```
### Multiple Parameters
```python
@pytest.mark.parametrize("a,b,expected", [
(2, 3, 5),
(0, 0, 0),
(-1, 1, 0),
(100, 200, 300),
])
def test_add(a, b, expected):
"""Test addition with multiple inputs."""
assert add(a, b) == expected
```
### Parametrize with IDs
```python
@pytest.mark.parametrize("input,expected", [
("valid@email.com", True),
("invalid", False),
("@no-domain.com", False),
], ids=["valid-email", "missing-at", "missing-domain"])
def test_email_validation(input, expected):
"""Test email validation with readable test IDs."""
assert is_valid_email(input) is expected
```
### Parametrized Fixtures
```python
@pytest.fixture(params=["sqlite", "postgresql", "mysql"])
def db(request):
"""Test against multiple database backends."""
if request.param == "sqlite":
return Database(":memory:")
elif request.param == "postgresql":
return Database("postgresql://localhost/test")
elif request.param == "mysql":
return Database("mysql://localhost/test")
def test_database_operations(db):
"""Test runs 3 times, once for each database."""
result = db.query("SELECT 1")
assert result is not None
```
## Markers and Test Selection
### Custom Markers
```python
# Mark slow tests
@pytest.mark.slow
def test_slow_operation():
time.sleep(5)
# Mark integration tests
@pytest.mark.integration
def test_api_integration():
response = requests.get("https://api.example.com")
assert response.status_code == 200
# Mark unit tests
@pytest.mark.unit
def test_unit_logic():
assert calculate(2, 3) == 5
```
### Run Specific Tests
```bash
# Run only fast tests
pytest -m "not slow"
# Run only integration tests
pytest -m integration
# Run integration or slow tests
pytest -m "integration or slow"
# Run tests marked as unit but not slow
pytest -m "unit and not slow"
```
### Configure Markers in pytest.ini
```ini
[pytest]
markers =
slow: marks tests as slow
integration: marks tests as integration tests
unit: marks tests as unit tests
django: marks tests as requiring Django
```
## Mocking and Patching
### Mocking Functions
```python
from unittest.mock import patch, Mock
@patch("mypackage.external_api_call")
def test_with_mock(api_call_mock):
"""Test with mocked external API."""
api_call_mock.return_value = {"status": "success"}
result = my_function()
api_call_mock.assert_called_once()
assert result["status"] == "success"
```
### Mocking Return Values
```python
@patch("mypackage.Database.connect")
def test_database_connection(connect_mock):
"""Test with mocked database connection."""
connect_mock.return_value = MockConnection()
db = Database()
db.connect()
connect_mock.assert_called_once_with("localhost")
```
### Mocking Exceptions
```python
@patch("mypackage.api_call")
def test_api_error_handling(api_call_mock):
"""Test error handling with mocked exception."""
api_call_mock.side_effect = ConnectionError("Network error")
with pytest.raises(ConnectionError):
api_call()
api_call_mock.assert_called_once()
```
### Mocking Context Managers
```python
@patch("builtins.open", new_callable=mock_open)
def test_file_reading(mock_file):
"""Test file reading with mocked open."""
mock_file.return_value.read.return_value = "file content"
result = read_file("test.txt")
mock_file.assert_called_once_with("test.txt", "r")
assert result == "file content"
```
### Using Autospec
```python
@patch("mypackage.DBConnection", autospec=True)
def test_autospec(db_mock):
"""Test with autospec to catch API misuse."""
db = db_mock.return_value
db.query("SELECT * FROM users")
# This would fail if DBConnection doesn't have query method
db_mock.assert_called_once()
```
### Mock Class Instances
```python
class TestUserService:
@patch("mypackage.UserRepository")
def test_create_user(self, repo_mock):
"""Test user creation with mocked repository."""
repo_mock.return_value.save.return_value = User(id=1, name="Alice")
service = UserService(repo_mock.return_value)
user = service.create_user(name="Alice")
assert user.name == "Alice"
repo_mock.return_value.save.assert_called_once()
```
### Mock Property
```python
@pytest.fixture
def mock_config():
"""Create a mock with a property."""
config = Mock()
type(config).debug = PropertyMock(return_value=True)
type(config).api_key = PropertyMock(return_value="test-key")
return config
def test_with_mock_config(mock_config):
"""Test with mocked config properties."""
assert mock_config.debug is True
assert mock_config.api_key == "test-key"
```
## Testing Async Code
### Async Tests with pytest-asyncio
```python
import pytest
@pytest.mark.asyncio
async def test_async_function():
"""Test async function."""
result = await async_add(2, 3)
assert result == 5
@pytest.mark.asyncio
async def test_async_with_fixture(async_client):
"""Test async with async fixture."""
response = await async_client.get("/api/users")
assert response.status_code == 200
```
### Async Fixture
```python
@pytest.fixture
async def async_client():
"""Async fixture providing async test client."""
app = create_app()
async with app.test_client() as client:
yield client
@pytest.mark.asyncio
async def test_api_endpoint(async_client):
"""Test using async fixture."""
response = await async_client.get("/api/data")
assert response.status_code == 200
```
### Mocking Async Functions
```python
@pytest.mark.asyncio
@patch("mypackage.async_api_call")
async def test_async_mock(api_call_mock):
"""Test async function with mock."""
api_call_mock.return_value = {"status": "ok"}
result = await my_async_function()
api_call_mock.assert_awaited_once()
assert result["status"] == "ok"
```
## Testing Exceptions
### Testing Expected Exceptions
```python
def test_divide_by_zero():
"""Test that dividing by zero raises ZeroDivisionError."""
with pytest.raises(ZeroDivisionError):
divide(10, 0)
def test_custom_exception():
"""Test custom exception with message."""
with pytest.raises(ValueError, match="invalid input"):
validate_input("invalid")
```
### Testing Exception Attributes
```python
def test_exception_with_details():
"""Test exception with custom attributes."""
with pytest.raises(CustomError) as exc_info:
raise CustomError("error", code=400)
assert exc_info.value.code == 400
assert "error" in str(exc_info.value)
```
## Testing Side Effects
### Testing File Operations
```python
import tempfile
import os
def test_file_processing():
"""Test file processing with temp file."""
with tempfile.NamedTemporaryFile(mode='w', delete=False, suffix='.txt') as f:
f.write("test content")
temp_path = f.name
try:
result = process_file(temp_path)
assert result == "processed: test content"
finally:
os.unlink(temp_path)
```
### Testing with pytest's tmp_path Fixture
```python
def test_with_tmp_path(tmp_path):
"""Test using pytest's built-in temp path fixture."""
test_file = tmp_path / "test.txt"
test_file.write_text("hello world")
result = process_file(str(test_file))
assert result == "hello world"
# tmp_path automatically cleaned up
```
### Testing with tmpdir Fixture
```python
def test_with_tmpdir(tmpdir):
"""Test using pytest's tmpdir fixture."""
test_file = tmpdir.join("test.txt")
test_file.write("data")
result = process_file(str(test_file))
assert result == "data"
```
## Test Organization
### Directory Structure
```
tests/
├── conftest.py # Shared fixtures
├── __init__.py
├── unit/ # Unit tests
│ ├── __init__.py
│ ├── test_models.py
│ ├── test_utils.py
│ └── test_services.py
├── integration/ # Integration tests
│ ├── __init__.py
│ ├── test_api.py
│ └── test_database.py
└── e2e/ # End-to-end tests
├── __init__.py
└── test_user_flow.py
```
### Test Classes
```python
class TestUserService:
"""Group related tests in a class."""
@pytest.fixture(autouse=True)
def setup(self):
"""Setup runs before each test in this class."""
self.service = UserService()
def test_create_user(self):
"""Test user creation."""
user = self.service.create_user("Alice")
assert user.name == "Alice"
def test_delete_user(self):
"""Test user deletion."""
user = User(id=1, name="Bob")
self.service.delete_user(user)
assert not self.service.user_exists(1)
```
## Best Practices
### DO
- **Follow TDD**: Write tests before code (red-green-refactor)
- **Test one thing**: Each test should verify a single behavior
- **Use descriptive names**: `test_user_login_with_invalid_credentials_fails`
- **Use fixtures**: Eliminate duplication with fixtures
- **Mock external dependencies**: Don't depend on external services
- **Test edge cases**: Empty inputs, None values, boundary conditions
- **Aim for 80%+ coverage**: Focus on critical paths
- **Keep tests fast**: Use marks to separate slow tests
### DON'T
- **Don't test implementation**: Test behavior, not internals
- **Don't use complex conditionals in tests**: Keep tests simple
- **Don't ignore test failures**: All tests must pass
- **Don't test third-party code**: Trust libraries to work
- **Don't share state between tests**: Tests should be independent
- **Don't catch exceptions in tests**: Use `pytest.raises`
- **Don't use print statements**: Use assertions and pytest output
- **Don't write tests that are too brittle**: Avoid over-specific mocks
## Common Patterns
### Testing API Endpoints (FastAPI/Flask)
```python
@pytest.fixture
def client():
app = create_app(testing=True)
return app.test_client()
def test_get_user(client):
response = client.get("/api/users/1")
assert response.status_code == 200
assert response.json["id"] == 1
def test_create_user(client):
response = client.post("/api/users", json={
"name": "Alice",
"email": "alice@example.com"
})
assert response.status_code == 201
assert response.json["name"] == "Alice"
```
### Testing Database Operations
```python
@pytest.fixture
def db_session():
"""Create a test database session."""
session = Session(bind=engine)
session.begin_nested()
yield session
session.rollback()
session.close()
def test_create_user(db_session):
user = User(name="Alice", email="alice@example.com")
db_session.add(user)
db_session.commit()
retrieved = db_session.query(User).filter_by(name="Alice").first()
assert retrieved.email == "alice@example.com"
```
### Testing Class Methods
```python
class TestCalculator:
@pytest.fixture
def calculator(self):
return Calculator()
def test_add(self, calculator):
assert calculator.add(2, 3) == 5
def test_divide_by_zero(self, calculator):
with pytest.raises(ZeroDivisionError):
calculator.divide(10, 0)
```
## pytest Configuration
### pytest.ini
```ini
[pytest]
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*
addopts =
--strict-markers
--disable-warnings
--cov=mypackage
--cov-report=term-missing
--cov-report=html
markers =
slow: marks tests as slow
integration: marks tests as integration tests
unit: marks tests as unit tests
```
### pyproject.toml
```toml
[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]
addopts = [
"--strict-markers",
"--cov=mypackage",
"--cov-report=term-missing",
"--cov-report=html",
]
markers = [
"slow: marks tests as slow",
"integration: marks tests as integration tests",
"unit: marks tests as unit tests",
]
```
## Running Tests
```bash
# Run all tests
pytest
# Run specific file
pytest tests/test_utils.py
# Run specific test
pytest tests/test_utils.py::test_function
# Run with verbose output
pytest -v
# Run with coverage
pytest --cov=mypackage --cov-report=html
# Run only fast tests
pytest -m "not slow"
# Run until first failure
pytest -x
# Run and stop on N failures
pytest --maxfail=3
# Run last failed tests
pytest --lf
# Run tests with pattern
pytest -k "test_user"
# Run with debugger on failure
pytest --pdb
```
## Quick Reference
| Pattern | Usage |
|---------|-------|
| `pytest.raises()` | Test expected exceptions |
| `@pytest.fixture()` | Create reusable test fixtures |
| `@pytest.mark.parametrize()` | Run tests with multiple inputs |
| `@pytest.mark.slow` | Mark slow tests |
| `pytest -m "not slow"` | Skip slow tests |
| `@patch()` | Mock functions and classes |
| `tmp_path` fixture | Automatic temp directory |
| `pytest --cov` | Generate coverage report |
| `assert` | Simple and readable assertions |
**Remember**: Tests are code too. Keep them clean, readable, and maintainable. Good tests catch bugs; great tests prevent them.

78
.env.example Normal file
View File

@@ -0,0 +1,78 @@
# 企微群机器人 webhook早报推送与 bot API 模式凭证不同)
WECOM_WEBHOOK_KEY=your-webhook-key
# 早报内容
DAILY_TRENDING_LIMIT=150
DAILY_HOT_LIMIT=150
DAILY_GITHUB_TRENDING_LIMIT=10
DAILY_GITHUB_EMERGING_LIMIT=10
DAILY_GITHUB_TOPIC_LIMIT=10
# GitHub Trending
GITHUB_TRENDING_SINCE=daily
# GITHUB_TRENDING_LANGUAGE=python
# GITHUB_TRENDING_MODE=scrape
# scrape — 爬 github.com/trending失败时用 token + Search API 补充
# api — 仅 Search API必须 GITHUB_TOKEN
# GitHub 新兴 / TopicSearch API需 GITHUB_TOKEN
# GITHUB_EMERGING_DAYS=14
# GITHUB_EMERGING_MIN_STARS=200
# GITHUB_TOPIC=llm
# GITHUB_TOPIC_PUSHED_DAYS=7
# GITHUB_TOPIC_MIN_STARS=50
# GITHUB_TOKEN=ghp_xxxxxxxxxxxxxxxx
# GITHUB_API_ENRICH=1
# 企微短版(各榜 Top N默认 10
DAILY_WECOM_TRENDING=10
DAILY_WECOM_HOT=10
DAILY_WECOM_GITHUB_TRENDING=10
DAILY_WECOM_GITHUB_EMERGING=10
DAILY_WECOM_GITHUB_TOPIC=10
DAILY_WECOM_AI_NEWS=10
# 企微 Skills 合并前扫描池大小(同 source 合并后仍凑满 Top N
# DAILY_WECOM_SKILL_POOL=200
# Skills 榜单来源website=skills.sh 官网600+ 条)| feed=第三方 feed.json50 条)
# SKILLS_BOARD_SOURCE=website
# 企微单条 markdown 上限 4096超长自动分多条推送不截断正文
DAILY_WECOM_CHUNK_BYTES=4096
# DAILY_WECOM_MAX_PARTS=5
# 国际 AI 时讯RSS见 daily/news/feeds.py
DAILY_AI_NEWS=1
# 英文描述 → 简短中文DAILY_CURSOR_EDITOR=0 时生效)
# DAILY_ZH_DESC=1
# DAILY_ZH_DESC_BATCH=20
# DAILY_LLM_API_KEY=
# DAILY_LLM_API_BASE=https://api.openai.com/v1
# DAILY_LLM_MODEL=gpt-4o-mini
# Tier BCursor 编辑层(主题 + 速览 + 中文描述,一次 LLM 调用)
# 需 CURSOR_API_KEY 或 DAILY_LLM_API_KEY开启后 DAILY_ZH_DESC 自动跳过
# DAILY_CURSOR_EDITOR=1
# Agent 工作流(推荐:叙事化早报)
# DAILY_REPORT_MODE=agent
# CURSOR_API_KEY=cursor_...
# CURSOR_MODEL=composer-2.5
# DAILY_CURSOR_CWD=d:\LY\diy\skills-hot-daily
# 新增榜对比(较昨日 Top15供 Agent 导语/signals列表展示 Top N
# DAILY_DELTA_COMPARE_DEPTH=15
# DAILY_WECOM_NEW_MAX=10
# DAILY_DELTA_LOOKBACK_DAYS=7
# DAILY_FULL_DESC_LIMIT=0
# DAILY_FULL_NEWS_SUMMARY_LIMIT=0
DAILY_AI_NEWS_HOURS=72
DAILY_AI_NEWS_PER_FEED=3
DAILY_AI_NEWS_PER_CATEGORY=5
# Reddit RSS403/429 时在 Reddit 偏好设置 → RSS feeds 复制 user / feed 参数)
# REDDIT_RSS_USER=your_username
# REDDIT_RSS_FEED=your_feed_token
# 可选:关注仓库 Release
# GITHUB_REPOS=vercel-labs/skills,obra/superpowers

8
.gitignore vendored Normal file
View File

@@ -0,0 +1,8 @@
.env
.env.local
logs/
.cache/
bot/.env
bot/.venv/
bot/.cache/
output

3
.idea/.gitignore generated vendored Normal file
View File

@@ -0,0 +1,3 @@
# Default ignored files
/shelf/
/workspace.xml

9
.idea/daily-robots.iml generated Normal file
View File

@@ -0,0 +1,9 @@
<?xml version="1.0" encoding="UTF-8"?>
<module type="JAVA_MODULE" version="4">
<component name="NewModuleRootManager" inherit-compiler-output="true">
<exclude-output />
<content url="file://$MODULE_DIR$" />
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
</component>
</module>

6
.idea/misc.xml generated Normal file
View File

@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ProjectRootManager" version="2" languageLevel="JDK_21" default="true" project-jdk-name="21" project-jdk-type="JavaSDK">
<output url="file://$PROJECT_DIR$/out" />
</component>
</project>

8
.idea/modules.xml generated Normal file
View File

@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ProjectModuleManager">
<modules>
<module fileurl="file://$PROJECT_DIR$/.idea/daily-robots.iml" filepath="$PROJECT_DIR$/.idea/daily-robots.iml" />
</modules>
</component>
</project>

6
.idea/vcs.xml generated Normal file
View File

@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="VcsDirectoryMappings">
<mapping directory="" vcs="Git" />
</component>
</project>

329
README.md Normal file
View File

@@ -0,0 +1,329 @@
# skills-hot-daily
Skills / GitHub 早报推送 + 企微对话机器人(同一仓库、两套企微接入)。
## 项目结构
```
skills-hot-daily/
├── README.md
├── .env.example # 早报 webhook、GitHub 等
├── requirements.txt # 早报 Python 依赖
├── run-daily.ps1 # 生成 + 推送一条龙
├── send-wecom.ps1 # 仅推送
├── daily/ # 早报 Python 包
│ ├── __main__.py # python -m daily [generate|push]
│ ├── config.py
│ ├── generate.py
│ ├── report_data.py # JSON 中间层
│ ├── cursor_editor.py # Cursor 编辑层
│ ├── llm_client.py
│ ├── localize.py # 仅中文化Tier A
│ ├── format_wecom.py
│ ├── webhook.py
│ ├── news/ # 国际 AI 时讯 RSS
│ │ ├── feeds.py
│ │ └── fetch.py
│ └── github/
│ ├── auth.py
│ ├── search.py
│ └── trending.py
├── output/ # YYYY-MM-DD.md / .wecom.md / .data.json / .editorial.json
├── skills/daily-editor/ # 早报 Cursor 编辑 Skill
│ └── SKILL.md
├── logs/
├── .cache/
└── bot/ # 企微 API 模式对话机器人(独立 venv
├── main.py
├── skills_service.py
└── scenarios/
```
| 模块 | 配置文件 | 启动方式 |
|------|----------|----------|
| **早报推送** | 根目录 `.env``WECOM_WEBHOOK_KEY` 等) | `.\run-daily.ps1` |
| **对话 Bot** | `bot/.env``WECOM_BOT_ID` / `SECRET` 等) | `cd bot``python main.py` |
---
## 一、早报推送
```powershell
cd d:\LY\test\tech\skills-hot-daily
pip install -r requirements.txt
copy .env.example .env
.\run-daily.ps1
```
- 生成:`output/YYYY-MM-DD.md`(完整版)、`output/YYYY-MM-DD.wecom.md`(企微短版)
- 仅生成:`.\run-daily.ps1 -SkipPush`
- 仅推送:`python -m daily push output\2026-06-25.wecom.md`
**Webhook 配置**:企微群 → 群机器人 → 添加,将 `key=` 后的值写入项目根 `.env`
```env
WECOM_WEBHOOK_KEY=your-key
```
**GitHub 数据源**(见 `.env.example`
| 来源 | 说明 |
|------|------|
| GitHub Trending | `GITHUB_TRENDING_MODE=scrape``api` |
| 新兴 / Topic | Search API`GITHUB_TOKEN` |
| Release | 可选 `GITHUB_REPOS=owner/repo` |
**国际 AI 时讯**RSS`daily/news/feeds.py`
| 类别 | 覆盖 |
|------|------|
| 厂商官方 | Anthropic、OpenAI、Google、Meta、Microsoft、Mistral、Cursor 等 |
| Agent / LLM 开发者 | LangChain、LlamaIndex、Hugging Face、Copilot 等 |
| 综合科技媒体 | The Verge、TechCrunch、Ars、Wired、MIT TR 等 |
| Newsletter | Ben's Bites、Latent Space、Simon Willison、TLDR AI 等 |
| 研究 / 论文 | arXiv cs.CL/AI/LG、HF Papers |
| 社区讨论 | HN、Reddit r/LocalLLaMA / ClaudeAI / ML 等 |
环境变量:`DAILY_AI_NEWS=1` · `DAILY_AI_NEWS_HOURS=72` · `DAILY_WECOM_AI_NEWS=8`
### 生成架构Tier B · Cursor 编辑层)
早报默认走 **Python 抓取 + 模板渲染**;可选开启 Cursor 做「编辑」:
```
抓取数据 → output/日期.data.json → Cursor 读 Skill 写 editorial → 模板填字 → .md / .wecom.md
```
| 文件 | 说明 |
|------|------|
| `output/YYYY-MM-DD.data.json` | 结构化榜单(供 LLM 输入) |
| `output/YYYY-MM-DD.editorial.json` | Cursor 输出的主题、速览、中文描述 |
| `skills/daily-editor/SKILL.md` | 编辑规范语气、JSON 格式) |
```env
# 开启 Tier B需 CURSOR_API_KEY 或 DAILY_LLM_API_KEY
DAILY_CURSOR_EDITOR=1
```
- 开启后:**一次 LLM 调用** 生成 `theme_line` + `highlights` + 全部中文描述
- 关闭时(默认):规则主题 + `DAILY_ZH_DESC` 仅中文化描述
- LLM 失败自动回退规则模式,不影响推送
### 生成架构Agent 工作流 · 推荐)
若觉得模板版「榜单堆砌」不友好,可改用 **Agent 三步流水线**
```
Python 抓取 → Step1 趋势分析 → Step2 叙事写稿 → Python 分条推送企微
(.trends.json) (.wecom.md)
```
| 模式 | 环境变量 | 企微版风格 |
|------|----------|------------|
| `classic`(默认) | — | 分区榜单 + 模板 |
| `editor` | `DAILY_CURSOR_EDITOR=1` | 模板 + LLM 中文化 |
| `agent` | `DAILY_REPORT_MODE=agent` | **导语 + 信号 + 精选**,全中文叙述 |
```env
DAILY_REPORT_MODE=agent
DAILY_CURSOR_CWD=d:\LY\diy\skills-hot-daily # 早报 LLM 工作目录(与 bot 的 CURSOR_CWD 独立)
```
| 文件 | 说明 |
|------|------|
| `output/YYYY-MM-DD.trends.json` | Step1 趋势分析结果 |
| `skills/daily-agent/SKILL.md` | Agent 工作流规范 |
- 完整版 `YYYY-MM-DD.md` 仍为数据表格归档;企微版由 Agent 直接写 Markdown
- Agent 失败自动回退 `classic`,不影响 `run-daily.ps1`
定时推送Windows 任务计划程序或 `/loop 1d` 执行 `run-daily.ps1`
---
## 二、可对话 Skills 助手(企业微信智能机器人)
在企微里 @ 机器人即可:
- **快查**`trending 10``hot 10``搜索 react`(本地 skills 数据,秒回)
- **截图预览**`preview` / `截图`(基于 `.env``CURSOR_CWD` 启动前端并发图)
- **通用任务**:任意自然语言需求,由 **Cursor Agent** 执行并回传结果
### 1. 创建 API 模式机器人
1. [企业微信管理后台](https://work.weixin.qq.com/) → **安全与管理****管理工具****智能机器人****创建机器人**
2. 选择 **API 模式创建****使用长连接**
3. 记录 **Bot ID****Secret**Secret 只显示一次,请立即保存)
4. 设置可见范围,将机器人 **添加到目标群** 或允许成员单聊
普通成员路径:工作台 → 智能机器人 → 手动创建 → API 模式 → 长连接
### 2. 启动本地服务
```powershell
cd d:\LY\test\tech\skills-hot-daily\bot
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
playwright install chromium
copy .env.example .env
# 编辑 .envWECOM_BOT_ID / WECOM_BOT_SECRET / CURSOR_API_KEY
python main.py
```
服务需 **常驻运行**(本机、服务器或 Docker。长连接模式下机器人进程须在线才能收消息。
### 3. 路由模式ROUTING_MODE
| 模式 | 行为 |
|------|------|
| `hybrid`(默认) | `trending`/`hot`/`搜索`/`详情` 走本地快查;其余 @ 消息交给 Cursor |
| `cursor` | 所有消息都交给 Cursor 执行 |
| `skills` | 仅本地 skills 快查(旧行为) |
**Cursor 任务示例**(群里发送):
```
@test 总结 trending top10并推荐 3 个适合前端团队的 skill
@test 对比 mattpocock/skills 和 obra/superpowers 各有哪些热门 skill
@test 帮我写一段 npx skills add 的安装说明
```
Cursor 在本机 `CURSOR_CWD` 目录下运行,默认 `d:\LY\test\tech`。复杂任务可能需要 110 分钟流式消息会显示「Cursor 正在执行任务…」。
### 4. 前端截图预览API 模式发图)
项目路径读取 `.env` 中的 **`CURSOR_CWD`**。机器人会:
1.`CURSOR_CWD` 检测 `package.json`,若有 `dev` 脚本则执行 `PREVIEW_DEV_COMMAND`(默认 `npm run dev`
2. 等待 `PREVIEW_PORT`(默认 `5173`)就绪,或用 `PREVIEW_URL` 直接访问
3. Playwright 打开页面并截图
4. 通过 API 模式 **上传图片 + 回复 image 消息** 到群
| 命令 | 说明 |
|------|------|
| `preview` / `截图` / `预览` | 访问 `http://127.0.0.1:5173/` 并截图 |
| `preview /login` | 指定路径 |
| `preview / 3000` | 指定端口 |
| `preview http://127.0.0.1:8080/` | 指定完整 URL |
**多步网页操作**(登录、点菜单、再截图)见下一节,不再写死在代码里。
`.env` 可选配置:
```env
CURSOR_CWD=d:\LY\test\tech
PREVIEW_PORT=5173
PREVIEW_URL=http://127.0.0.1:5173/
PREVIEW_DEV_COMMAND=npm run dev
PREVIEW_STARTUP_TIMEOUT=120
```
`CURSOR_CWD` 下暂无前端项目,可先手动启动 dev server或设置 `PREVIEW_URL` 指向已运行地址。
### 4b. 网页操作Playwright 步骤引擎)
支持三种方式定义操作流程,**无需改 Python 代码**
**1. 自然语言(企微里直接说)**
```
@test 访问登录页,输入账号密码,点击登录后进入主页,点击智能体管理菜单然后截图
```
账号密码从 `.env` 读取(`{{PREVIEW_LOGIN_USER}}` / `{{PREVIEW_LOGIN_PASSWORD}}`),勿在群里发密码。
**2. 场景文件 YAML**
`bot/scenarios/xiaobao-agent-manage.yaml` 示例:
```yaml
name: xiaobao-agent-manage
steps:
- goto: /login
- fill:
field: 账号
value: "{{PREVIEW_LOGIN_USER}}"
- fill:
field: 密码
value: "{{PREVIEW_LOGIN_PASSWORD}}"
- click: 登录
- wait:
url: "**/app/**"
- click: 智能体管理
- wait: 1500
- screenshot
```
触发:`@test browser xiaobao-agent-manage`
场景搜索路径:`bot/scenarios/``CURSOR_CWD/.browser-scenarios/`、环境变量 `BROWSER_SCENARIOS_DIR`
**3. 消息内 DSL**
```
browser:
goto /login
fill 账号 {{PREVIEW_LOGIN_USER}}
fill 密码 {{PREVIEW_LOGIN_PASSWORD}}
click 登录
click 智能体管理
screenshot
```
**支持的步骤**`goto` · `fill` · `click` · `wait` · `screenshot` · `press`
`.env` 登录与场景配置:
```env
PREVIEW_LOGIN_USER=test_account
PREVIEW_LOGIN_PASSWORD=your_password
# BROWSER_DEFAULT_SCENARIO=xiaobao-agent-manage
```
### 5. 支持的快查命令
| 命令 | 说明 |
|------|------|
| `trending 10` / `趋势 10` | 近期增长榜 Top N默认 10最大 30 |
| `hot 10` / `实时 10` | 实时热度榜 |
| `all 10` / `总榜 10` | 历史总安装榜 |
| `搜索 react` / `search tdd` | 关键词搜索 |
| `详情 find-skills` | 单个 skill 详情 + 安装命令 |
| `preview` / `截图` | 启动 CURSOR_CWD 前端并截图发群 |
| `帮助` | 命令列表 |
自然语言(非显式快查命令)会交给 **Cursor** 处理,例如 `@test 查 trending 并写推荐`
### 6. 本地测试(无需企微凭证)
```powershell
cd d:\LY\test\tech\skills-hot-daily\bot
python -c "from skills_service import handle_command; print(handle_command('trending 5'))"
```
---
## 其他推送方案
| 方案 | 适用场景 | 复杂度 |
|------|----------|--------|
| **群机器人 webhook** | 推送到固定群 | 低 |
| **应用消息 API** | 推送给指定成员/部门 | 中(需 corp_id、secret、agent_id |
| **邮件 + 企业微信邮箱** | 已有 SMTP | 中 |
| **PushPlus / Server酱** | 个人微信中转 | 低(第三方) |
### 应用消息 API简要
适合「推送给某个人」而非群聊。需在 [企业微信管理后台](https://work.weixin.qq.com/) 创建自建应用,调用:
`POST https://qyapi.weixin.qq.com/cgi-bin/message/send?access_token=TOKEN`
消息体支持 `text` / `markdown` / `news` 等。需先 `gettoken` 再发消息,并维护 access_token 缓存。
---
## 注意事项
- Webhook **不要提交到 Git**,只用环境变量
- 企业微信 markdown 为**子集**(不支持完整 GitHub 表格语法时可改为文本列表)
- 单条消息约 **4096 字节** 上限,`send-wecom.ps1` 已做截断

29
bot/.env.example Normal file
View File

@@ -0,0 +1,29 @@
# 企业微信智能机器人API 模式 · 长连接)
# 管理后台 → 安全与管理 → 管理工具 → 智能机器人 → 创建 → API 模式 → 使用长连接
WECOM_BOT_ID=your-bot-id
WECOM_BOT_SECRET=your-bot-secret
# Cursor SDK@ 机器人后的通用任务由 Cursor 执行)
CURSOR_API_KEY=cursor_...
CURSOR_CWD=d:\LY\test\tech
CURSOR_MODEL=composer-2.5
CURSOR_TIMEOUT=600
# 前端截图预览(基于 CURSOR_CWD
PREVIEW_PORT=5173
PREVIEW_URL=http://127.0.0.1:5173/
# PREVIEW_DEV_COMMAND=npm run dev
# PREVIEW_STARTUP_TIMEOUT=120
# 登录后截图(账号密码只放 .env切勿发到企微群
# PREVIEW_LOGIN_USER=your_account_or_phone
# PREVIEW_LOGIN_PASSWORD=your_password
# PREVIEW_AFTER_LOGIN_URL=/app/dashboard
# PREVIEW_AUTO_LOGIN=true
# 网页操作场景目录(可选,默认 bot/scenarios 与 CURSOR_CWD/.browser-scenarios
# BROWSER_SCENARIOS_DIR=d:\path\to\scenarios
# BROWSER_DEFAULT_SCENARIO=xiaobao-agent-manage
# hybrid=快查走本地 / 其余走 Cursor | cursor=全部 Cursor | skills=仅本地
ROUTING_MODE=hybrid

4
bot/.gitignore vendored Normal file
View File

@@ -0,0 +1,4 @@
.cache/
.env
.venv/
.cache/screenshots/

12
bot/bot_types.py Normal file
View File

@@ -0,0 +1,12 @@
"""Bot 内部数据结构。"""
from __future__ import annotations
from dataclasses import dataclass
@dataclass
class RouteResult:
source: str
text: str
image_path: str | None = None

156
bot/bridge_manager.py Normal file
View File

@@ -0,0 +1,156 @@
"""Windows 兼容的 Cursor SDK bridge 管理。"""
from __future__ import annotations
import codecs
import json
import logging
import os
import subprocess
import threading
import time
from pathlib import Path
from typing import Any, Mapping
import env_config
logger = logging.getLogger(__name__)
READY_LINE_PREFIX = "cursor-sdk-bridge ready "
_bridge_lock = threading.Lock()
_bridge_process: subprocess.Popen[bytes] | None = None
def _cursor_cwd() -> str:
return env_config.env("CURSOR_CWD", r"d:\LY\test\tech") or r"d:\LY\test\tech"
def _parse_discovery_line(line: str) -> Mapping[str, Any] | None:
if not line.startswith(READY_LINE_PREFIX):
return None
payload = line[len(READY_LINE_PREFIX) :].strip()
loaded = json.loads(payload)
if not isinstance(loaded, dict):
raise RuntimeError("Bridge discovery payload must be an object")
return loaded
def _read_discovery_polling(process: subprocess.Popen[bytes], timeout: float = 60) -> Mapping[str, Any]:
"""不用 selectors避免 Windows 上 WinError 10038。"""
if process.stderr is None:
raise RuntimeError("Bridge stderr unavailable")
fd = process.stderr.fileno()
was_blocking = os.get_blocking(fd)
os.set_blocking(fd, False)
decoder = codecs.getincrementaldecoder("utf-8")(errors="replace")
pending = ""
stderr_lines: list[str] = []
deadline = time.monotonic() + timeout
try:
while time.monotonic() < deadline:
try:
chunk = os.read(fd, 8192)
except BlockingIOError:
chunk = b""
if chunk:
pending += decoder.decode(chunk)
while "\n" in pending:
line, pending = pending.split("\n", 1)
stderr_lines.append(line)
discovery = _parse_discovery_line(line)
if discovery is not None:
return discovery
else:
code = process.poll()
if code is not None:
pending += decoder.decode(b"", final=True)
if pending.strip():
stderr_lines.append(pending.strip())
joined = "\n".join(stderr_lines)[-2000:]
raise RuntimeError(
f"Bridge 启动失败 exit={code}: {joined or '无 stderr 输出'}"
)
time.sleep(0.05)
finally:
os.set_blocking(fd, was_blocking)
raise RuntimeError("等待 Cursor bridge 就绪超时")
def _auth_token_from_discovery(discovery: Mapping[str, Any]) -> str:
token = str(discovery.get("authToken") or "").strip()
if token:
return token
token_file = discovery.get("authTokenFile")
if token_file:
return Path(str(token_file)).read_text(encoding="utf-8").strip()
raise RuntimeError("Bridge discovery 缺少 auth token")
def warm_cursor_bridge(force: bool = False) -> None:
"""启动 cursor-sdk-bridge 并写入 CURSOR_SDK_BRIDGE_* 环境变量。"""
global _bridge_process
with _bridge_lock:
if (
not force
and _bridge_process is not None
and _bridge_process.poll() is None
and os.environ.get("CURSOR_SDK_BRIDGE_URL")
and os.environ.get("CURSOR_SDK_BRIDGE_TOKEN")
):
return
if _bridge_process is not None and _bridge_process.poll() is None:
_bridge_process.terminate()
try:
_bridge_process.wait(timeout=5)
except subprocess.TimeoutExpired:
_bridge_process.kill()
from cursor_sdk._vendor import resolve_bridge_path
cwd = _cursor_cwd()
argv = [resolve_bridge_path(), "--workspace", cwd]
logger.info("启动 Cursor bridge workspace=%s", cwd)
process = subprocess.Popen(
argv,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
)
try:
discovery = _read_discovery_polling(process)
except Exception:
process.kill()
process.wait(timeout=5)
raise
url = str(discovery.get("url") or "").strip()
if not url:
host = str(discovery.get("host") or "127.0.0.1")
port = discovery.get("port")
url = f"http://{host}:{port}"
token = _auth_token_from_discovery(discovery)
os.environ["CURSOR_SDK_BRIDGE_URL"] = url
os.environ["CURSOR_SDK_BRIDGE_TOKEN"] = token
_bridge_process = process
logger.info("Cursor bridge 就绪: %s", url)
def shutdown_cursor_bridge() -> None:
global _bridge_process
with _bridge_lock:
if _bridge_process is None:
return
if _bridge_process.poll() is None:
_bridge_process.terminate()
try:
_bridge_process.wait(timeout=5)
except subprocess.TimeoutExpired:
_bridge_process.kill()
_bridge_process = None

49
bot/browser_env.py Normal file
View File

@@ -0,0 +1,49 @@
"""浏览器场景变量替换与 base URL 解析。"""
from __future__ import annotations
import re
from urllib.parse import urlparse
import env_config
_VAR_PATTERN = re.compile(r"\{\{([A-Z0-9_]+)\}\}")
def interpolate(value: str) -> str:
def repl(match: re.Match[str]) -> str:
key = match.group(1)
resolved = env_config.env(key)
if resolved is None:
raise RuntimeError(f"场景变量未配置:{key}")
return resolved
return _VAR_PATTERN.sub(repl, value)
def default_base_url() -> str:
explicit = (env_config.env("PREVIEW_BASE_URL") or "").strip()
if explicit:
return interpolate(explicit.rstrip("/"))
preview = (env_config.env("PREVIEW_URL") or "").strip()
if preview:
parsed = urlparse(preview)
scheme = parsed.scheme or "http"
host = parsed.hostname or "127.0.0.1"
port = parsed.port
if port and port not in (80, 443):
return f"{scheme}://{host}:{port}"
return f"{scheme}://{host}"
port = env_config.env("PREVIEW_PORT", "5173") or "5173"
return f"http://127.0.0.1:{port}"
def resolve_url(base_url: str, target: str) -> str:
target = interpolate(target.strip())
if target.startswith("http://") or target.startswith("https://"):
return target
if not target.startswith("/"):
target = "/" + target
return base_url.rstrip("/") + target

269
bot/browser_executor.py Normal file
View File

@@ -0,0 +1,269 @@
"""通用 Playwright 步骤执行器(不写死业务页面)。"""
from __future__ import annotations
import logging
import re
import time
from datetime import datetime
from pathlib import Path
from typing import Any
from browser_env import interpolate, resolve_url
from browser_models import BrowserResult, BrowserScenario
logger = logging.getLogger(__name__)
SCREENSHOT_DIR = Path(__file__).resolve().parent / ".cache" / "screenshots"
FIELD_HINTS: dict[str, list[str]] = {
"账号": [
"#login-username",
"input#login-username",
"input[autocomplete='username']",
"username",
"account",
"phone",
"账号",
"手机号",
"企业账号",
],
"密码": [
"#login-password input",
"#login-password",
"input#login-password",
"input[type='password']",
"password",
"密码",
],
"用户名": ["#login-username", "input#login-username", "username", "account", "账号"],
}
def _step_label(step: dict[str, Any], index: int) -> str:
action = step.get("action", "?")
target = step.get("target") or step.get("field") or step.get("url") or ""
return f"{index + 1}. {action} {target}".strip()
def _resolve_fill_locator(page, field: str, step: dict[str, Any]):
if step.get("selector"):
return page.locator(interpolate(str(step["selector"])))
field_key = interpolate(str(field))
if step.get("label"):
return page.get_by_label(interpolate(str(step["label"])), exact=False)
if step.get("placeholder"):
return page.get_by_placeholder(interpolate(str(step["placeholder"])), exact=False)
hints = FIELD_HINTS.get(field_key, [field_key])
for hint in hints:
if hint.startswith("#") or hint.startswith(".") or hint.startswith("["):
locator = page.locator(hint)
if locator.count() > 0:
return locator.first
for getter in (
lambda h=hint: page.get_by_label(h, exact=False),
lambda h=hint: page.get_by_placeholder(h, exact=False),
):
locator = getter()
if locator.count() > 0:
return locator.first
return page.locator("input, textarea").filter(has_text=field_key).first
def _fill_field(page, field: str, step: dict[str, Any]) -> None:
value = interpolate(str(step.get("value", "")))
locator = _resolve_fill_locator(page, field, step)
locator.click(timeout=10_000)
locator.fill("", timeout=5_000)
locator.fill(value, timeout=10_000)
def _page_error_text(page) -> str | None:
for selector in (
".ant-message-error",
".ant-form-item-explain-error",
".ant-alert-error",
):
try:
locator = page.locator(selector).first
if locator.is_visible(timeout=300):
text = locator.inner_text(timeout=1_000).strip()
if text:
return text
except Exception:
continue
return None
def _pathname_matches(pattern: str, pathname: str) -> bool:
pattern = pattern.strip()
if pattern in {"**/app/**", "**/app/*", "/app/**"}:
return pathname.startswith("/app")
if pattern.endswith("/**"):
prefix = pattern[:-3].rstrip("/")
if prefix.startswith("**/"):
prefix = prefix[3:]
if not prefix.startswith("/"):
prefix = "/" + prefix
return pathname.startswith(prefix)
if "**" in pattern or "*" in pattern:
regex = "^" + re.escape(pattern).replace(r"\*\*", ".*").replace(r"\*", "[^/]*") + "$"
return re.search(regex, pathname) is not None
return pathname == pattern or pathname.startswith(pattern)
def _wait_for_url_pattern(page, pattern: str, timeout: int = 60_000) -> None:
"""SPA 路由用 pathname 轮询glob 模式不依赖 navigation 事件。"""
deadline = time.monotonic() + timeout / 1000
last_error: str | None = None
while time.monotonic() < deadline:
pathname = page.evaluate("() => window.location.pathname")
if _pathname_matches(pattern, pathname):
try:
page.wait_for_load_state("networkidle", timeout=8_000)
except Exception:
page.wait_for_timeout(800)
return
err = _page_error_text(page)
if err and err != last_error:
last_error = err
logger.warning("页面提示:%s", err)
if "/login" in pathname:
raise RuntimeError(f"登录失败:{err}")
page.wait_for_timeout(400)
err = _page_error_text(page)
hint_parts = [f"当前 URL`{page.url}`"]
if err:
hint_parts.append(f"页面错误:{err}")
elif last_error:
hint_parts.append(f"页面错误:{last_error}")
hint_parts.append("请确认 PREVIEW_LOGIN_USER/PASSWORD 正确,且登录 API内网网关可达。")
raise RuntimeError(f"等待 URL 匹配 `{pattern}` 超时({timeout}ms{' '.join(hint_parts)}")
def _click_target(page, target: str) -> None:
target = interpolate(target.strip())
if target.lower() in {"登录", "login"}:
for selector in ("button.login-submit", "button[type='submit']"):
locator = page.locator(selector)
if locator.count() > 0:
locator.first.click(timeout=10_000)
return
candidates = [
page.get_by_role("menuitem", name=target, exact=True),
page.get_by_role("button", name=target, exact=True),
page.get_by_role("link", name=target, exact=True),
page.get_by_text(target, exact=True),
page.get_by_text(target, exact=False),
]
for locator in candidates:
if locator.count() > 0:
locator.first.click(timeout=10_000)
return
raise RuntimeError(f"未找到可点击元素:{target}")
def _execute_step(page, base_url: str, step: dict[str, Any]) -> None:
action = str(step.get("action", "")).lower()
if action == "goto":
target = step.get("target") or step.get("url") or "/"
url = resolve_url(base_url, str(target))
page.goto(url, wait_until="networkidle", timeout=60_000)
return
if action == "fill":
field = str(step.get("field") or step.get("target") or "账号")
_fill_field(page, field, step)
return
if action == "click":
target = step.get("target") or step.get("text")
if not target:
raise RuntimeError("click 步骤缺少 target")
_click_target(page, str(target))
page.wait_for_timeout(800)
return
if action == "wait":
timeout = int(step.get("timeout") or 60_000)
if step.get("url"):
_wait_for_url_pattern(page, str(step["url"]), timeout=timeout)
return
if step.get("selector"):
page.locator(interpolate(str(step["selector"]))).wait_for(timeout=30_000)
return
if step.get("text"):
page.get_by_text(interpolate(str(step["text"])), exact=False).wait_for(timeout=30_000)
return
ms = int(step.get("ms") or 1500)
page.wait_for_timeout(ms)
return
if action == "press":
key = str(step.get("key") or step.get("target") or "Enter")
page.keyboard.press(key)
return
if action == "screenshot":
return
raise RuntimeError(f"未知步骤 action={action}")
def run_browser_scenario_sync(scenario: BrowserScenario) -> BrowserResult:
from playwright.sync_api import sync_playwright
stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
slug = (scenario.name or "browser").replace(" ", "-")
output = SCREENSHOT_DIR / f"{slug}-{stamp}.png"
output.parent.mkdir(parents=True, exist_ok=True)
step_log: list[str] = []
final_url = scenario.base_url
with sync_playwright() as playwright:
browser = playwright.chromium.launch(headless=True)
page = browser.new_page(viewport={"width": 1280, "height": 720})
steps = list(scenario.steps)
if steps and steps[-1].get("action") != "screenshot" and not any(
s.get("action") == "screenshot" for s in steps
):
steps.append({"action": "screenshot"})
for index, step in enumerate(steps):
label = _step_label(step, index)
logger.info("执行步骤 %s", label)
action = str(step.get("action", "")).lower()
if action == "screenshot":
page.wait_for_timeout(int(step.get("ms") or 1500))
page.screenshot(path=str(output), full_page=False, type="png")
final_url = page.url
step_log.append(label + "")
continue
try:
_execute_step(page, scenario.base_url, step)
final_url = page.url
step_log.append(label + "")
except Exception as exc:
err = _page_error_text(page)
detail = f"{err}" if err else ""
raise RuntimeError(f"步骤失败:{label} @ {page.url}{detail}") from exc
browser.close()
return BrowserResult(
scenario_name=scenario.name,
base_url=scenario.base_url,
final_url=final_url,
screenshot_path=output,
step_count=len(steps),
step_log=step_log,
)

26
bot/browser_models.py Normal file
View File

@@ -0,0 +1,26 @@
"""浏览器自动化步骤模型。"""
from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
@dataclass
class BrowserScenario:
name: str | None
base_url: str
steps: list[dict[str, Any]]
source: str = "natural"
@dataclass
class BrowserResult:
scenario_name: str | None
base_url: str
final_url: str
screenshot_path: Path
step_count: int
started_dev_server: bool = False
step_log: list[str] = field(default_factory=list)

307
bot/browser_parser.py Normal file
View File

@@ -0,0 +1,307 @@
"""解析自然语言 / YAML / 场景名 → 浏览器步骤。"""
from __future__ import annotations
import re
from pathlib import Path
from typing import Any
import yaml
import env_config
from browser_env import default_base_url, interpolate
from browser_models import BrowserScenario
SCENARIO_DIRS = [
Path(__file__).resolve().parent / "scenarios",
Path(__file__).resolve().parent.parent / "scenarios",
]
def _project_cwd() -> Path:
raw = env_config.env("CURSOR_CWD", r"d:\LY\test\tech") or r"d:\LY\test\tech"
return Path(raw).resolve()
def _strip_mention(text: str) -> str:
return re.sub(r"@\S+\s*", "", text).strip()
def _scenario_search_dirs() -> list[Path]:
dirs = list(SCENARIO_DIRS)
dirs.append(_project_cwd() / ".browser-scenarios")
custom = (env_config.env("BROWSER_SCENARIOS_DIR") or "").strip()
if custom:
dirs.append(Path(custom).resolve())
return dirs
def is_browser_intent(text: str) -> bool:
raw = _strip_mention(text)
if not raw:
return False
if re.match(r"^(browser|网页|网页操作|操作)\b", raw, re.IGNORECASE):
return True
if re.search(r"```(?:yaml|yml)", raw, re.IGNORECASE):
return True
if re.search(r"(?m)^browser\s*:", raw, re.IGNORECASE):
return True
if re.match(r"^(preview|截图|预览|截屏)\s", raw, re.IGNORECASE):
if not re.search(r"[,。;;]|然后|输入|点击|填写|访问|打开|登录", raw):
return False
if len(_split_segments(text)) >= 2:
return True
verbs = 0
for pattern in (r"访问", r"打开", r"输入", r"填写", r"点击", r"点选", r"选择", r"登录"):
if re.search(pattern, raw):
verbs += 1
return verbs >= 2
def _load_yaml_scenario(path: Path) -> BrowserScenario:
data = yaml.safe_load(path.read_text(encoding="utf-8"))
if not isinstance(data, dict):
raise RuntimeError(f"场景文件格式错误:{path}")
base_url = interpolate(str(data.get("base_url") or default_base_url()))
steps = data.get("steps")
if not isinstance(steps, list) or not steps:
raise RuntimeError(f"场景缺少 steps{path}")
return BrowserScenario(
name=data.get("name") or path.stem,
base_url=base_url,
steps=_normalize_steps(steps),
source=f"file:{path.name}",
)
def _find_scenario_file(name: str) -> Path | None:
slug = name.strip().replace(" ", "-")
for directory in _scenario_search_dirs():
for candidate in (directory / f"{slug}.yaml", directory / f"{slug}.yml"):
if candidate.exists():
return candidate
return None
def _normalize_steps(raw_steps: list[Any]) -> list[dict[str, Any]]:
normalized: list[dict[str, Any]] = []
for item in raw_steps:
if isinstance(item, str):
normalized.append({"action": item})
continue
if not isinstance(item, dict) or not item:
raise RuntimeError(f"无效步骤:{item!r}")
if "action" in item:
normalized.append(dict(item))
continue
if len(item) == 1:
action, payload = next(iter(item.items()))
step = {"action": action}
if payload is not None:
if isinstance(payload, dict):
step.update(payload)
elif action == "wait" and isinstance(payload, int):
step["ms"] = payload
elif action == "wait" and isinstance(payload, str) and payload.isdigit():
step["ms"] = int(payload)
else:
step["target"] = payload
normalized.append(step)
continue
raise RuntimeError(f"无效步骤:{item!r}")
return normalized
def _parse_inline_dsl(text: str) -> BrowserScenario | None:
raw = _strip_mention(text)
match = re.search(r"(?ms)^browser\s*:\s*\n(.+)$", raw, re.IGNORECASE)
if not match:
return None
steps: list[dict[str, Any]] = []
for line in match.group(1).splitlines():
line = line.strip()
if not line or line.startswith("#"):
continue
line = re.sub(r"^[-*]\s*", "", line)
if not line:
continue
steps.append(_parse_dsl_line(line))
if not steps:
return None
return BrowserScenario(
name="inline",
base_url=default_base_url(),
steps=steps,
source="inline-dsl",
)
def _parse_dsl_line(line: str) -> dict[str, Any]:
parts = line.split(None, 2)
action = parts[0].lower()
if action == "goto":
return {"action": "goto", "target": parts[1] if len(parts) > 1 else "/"}
if action == "click":
return {"action": "click", "target": " ".join(parts[1:])}
if action == "fill":
if len(parts) < 3:
raise RuntimeError(f"fill 语法fill 字段 值({line}")
return {"action": "fill", "field": parts[1], "value": parts[2]}
if action == "wait":
payload = parts[1] if len(parts) > 1 else "1500"
if payload.isdigit():
return {"action": "wait", "ms": int(payload)}
return {"action": "wait", "url": payload}
if action in {"screenshot", "shot"}:
return {"action": "screenshot"}
raise RuntimeError(f"未知 DSL 步骤:{line}")
def _split_segments(text: str) -> list[str]:
raw = _strip_mention(text)
raw = re.sub(r"^(browser|网页|网页操作|操作)\s*[:]?\s*", "", raw, flags=re.IGNORECASE)
raw = re.sub(r"然后截图|再截图|最后截图", "截图", raw)
chunks = re.split(r"[,。;;]\s*|\s+然后\s+|\s+接着\s+|\s+并\s*", raw)
expanded: list[str] = []
for chunk in chunks:
chunk = chunk.strip()
if not chunk:
continue
subchunks = re.split(r"\s+然后\s+", chunk)
if "" in chunk and len(subchunks) == 1:
subchunks = re.split(r"(?<=[登录页表单])后(?=[进入打开等待点击访问])", chunk)
for part in subchunks:
part = part.strip()
if part:
expanded.append(part)
return expanded
def _parse_segment(segment: str) -> list[dict[str, Any]]:
seg = segment.strip()
if not seg or seg.lower() in {"browser", "网页操作"}:
return []
if re.fullmatch(r"截图|截屏", seg, re.IGNORECASE):
return [{"action": "screenshot"}]
match = re.search(r"访问登录页|打开登录页|进入登录页|运行登录页", seg, re.IGNORECASE)
if match:
return [{"action": "goto", "target": "/login"}]
match = re.search(r"输入账号密码|填写账号密码|输入账号和密码", seg, re.IGNORECASE)
if match:
return [
{"action": "fill", "field": "账号", "value": "{{PREVIEW_LOGIN_USER}}"},
{"action": "fill", "field": "密码", "value": "{{PREVIEW_LOGIN_PASSWORD}}"},
]
match = re.search(r"输入账号|填写账号|输入用户名|填写用户名", seg, re.IGNORECASE)
if match:
return [{"action": "fill", "field": "账号", "value": "{{PREVIEW_LOGIN_USER}}"}]
match = re.search(r"输入密码|填写密码", seg, re.IGNORECASE)
if match:
return [{"action": "fill", "field": "密码", "value": "{{PREVIEW_LOGIN_PASSWORD}}"}]
match = re.search(r"进入主页|进入首页|打开主页|打开首页|等待主页", seg, re.IGNORECASE)
if match:
return [{"action": "wait", "url": "**/app/**"}]
match = re.search(r"等待\s*(\d+)\s*秒", seg, re.IGNORECASE)
if match:
return [{"action": "wait", "ms": int(match.group(1)) * 1000}]
match = re.search(
r"(?:点击|点选|选择)\s*(.+?)(?:菜单|按钮|链接)?$",
seg,
re.IGNORECASE,
)
if match:
target = match.group(1).strip()
target = re.sub(r"(然后|再|并)?\s*(截图|截屏).*$", "", target, flags=re.IGNORECASE).strip()
target = re.sub(r"(然后|再|之后)$", "", target).strip()
target = re.sub(r"(菜单|按钮|链接)$", "", target).strip()
if target:
return [{"action": "click", "target": target}]
match = re.search(
r"(?:访问|打开|进入)\s*(https?://\S+|/\S+|登录页|主页|首页)",
seg,
re.IGNORECASE,
)
if match:
target = match.group(1)
mapping = {"登录页": "/login", "主页": "/app/dashboard", "首页": "/app/dashboard"}
return [{"action": "goto", "target": mapping.get(target, target)}]
return []
def parse_natural_language(text: str) -> BrowserScenario | None:
segments = _split_segments(text)
steps: list[dict[str, Any]] = []
for segment in segments:
steps.extend(_parse_segment(segment))
if not steps:
return None
if not any(step.get("action") == "screenshot" for step in steps):
if re.search(r"截图|截屏", text, re.IGNORECASE):
steps.append({"action": "screenshot"})
if not steps:
return None
return BrowserScenario(
name="natural",
base_url=default_base_url(),
steps=steps,
source="natural-language",
)
def parse_browser_request(text: str) -> BrowserScenario | None:
if not is_browser_intent(text):
return None
raw = _strip_mention(text)
yaml_block = re.search(r"```(?:yaml|yml)\s*\n(.+?)```", raw, re.IGNORECASE | re.DOTALL)
if yaml_block:
data = yaml.safe_load(yaml_block.group(1))
if isinstance(data, dict):
base_url = interpolate(str(data.get("base_url") or default_base_url()))
steps = data.get("steps") or []
return BrowserScenario(
name=data.get("name") or "yaml-inline",
base_url=base_url,
steps=_normalize_steps(steps),
source="yaml-inline",
)
inline = _parse_inline_dsl(text)
if inline:
return inline
match = re.match(r"^(browser|网页|网页操作|操作)\s+([\w\-./]+)\s*$", raw, re.IGNORECASE)
if match:
path = _find_scenario_file(match.group(2))
if not path:
raise RuntimeError(f"未找到场景文件:{match.group(2)}.yaml")
return _load_yaml_scenario(path)
scenario = parse_natural_language(text)
if scenario:
return scenario
default_name = (env_config.env("BROWSER_DEFAULT_SCENARIO") or "").strip()
if default_name:
path = _find_scenario_file(default_name)
if path:
return _load_yaml_scenario(path)
return None

87
bot/browser_service.py Normal file
View File

@@ -0,0 +1,87 @@
"""浏览器自动化服务:解析场景 + 启动 dev server + 执行步骤。"""
from __future__ import annotations
import asyncio
import logging
from urllib.parse import urlparse
from browser_executor import run_browser_scenario_sync
from browser_models import BrowserResult, BrowserScenario
from browser_parser import parse_browser_request
from preview_service import (
_package_dev_script,
_preview_port,
_project_cwd,
_startup_timeout,
_wait_for_port,
)
logger = logging.getLogger(__name__)
def _ensure_dev_server(base_url: str) -> bool:
parsed = urlparse(base_url)
host = parsed.hostname or "127.0.0.1"
port = parsed.port or (443 if parsed.scheme == "https" else 80)
if _wait_for_port(host, port, timeout=3):
return False
cwd = _project_cwd()
dev_command = _package_dev_script(cwd)
if not dev_command:
raise RuntimeError(
f"无法访问 {base_url},且未找到可启动的 dev 脚本。"
"请先手动启动前端,或设置 PREVIEW_URL。"
)
import subprocess
logger.info("启动 dev server: %s (cwd=%s)", dev_command, cwd)
proc = subprocess.Popen(
dev_command,
cwd=str(cwd),
shell=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
)
if not _wait_for_port(host, port, timeout=_startup_timeout()):
err = ""
if proc.stderr:
err = proc.stderr.read().decode("utf-8", errors="replace")[-1000:]
proc.kill()
raise RuntimeError(
f"dev server 在 {_startup_timeout()}s 内未就绪 ({base_url})。"
f"{(' 日志: ' + err) if err else ''}"
)
return True
async def run_browser_automation(text: str) -> BrowserResult:
scenario = parse_browser_request(text)
if scenario is None:
raise RuntimeError("无法解析网页操作步骤")
started = await asyncio.to_thread(_ensure_dev_server, scenario.base_url)
result = await asyncio.to_thread(run_browser_scenario_sync, scenario)
result.started_dev_server = started
return result
def format_browser_caption(result: BrowserResult) -> str:
lines = [
"**网页操作完成**",
f"> 场景:`{result.scenario_name or '自定义'}`",
f"> 起始:`{result.base_url}`",
f"> 最终:`{result.final_url}`",
f"> 步骤数:{result.step_count}",
]
if result.started_dev_server:
lines.append("> dev server已自动启动")
if result.step_log:
lines.append("")
lines.append("执行记录:")
for item in result.step_log[-8:]:
lines.append(f"- {item}")
return "\n".join(lines)

106
bot/cursor_runner.py Normal file
View File

@@ -0,0 +1,106 @@
"""通过 Cursor SDK 执行用户任务。"""
from __future__ import annotations
import asyncio
import logging
import re
from typing import Awaitable, Callable
import env_config
from bridge_manager import warm_cursor_bridge
logger = logging.getLogger(__name__)
_cursor_lock = asyncio.Lock()
WECHAT_SYSTEM_PREFIX = """你是企业微信群里的 Skills 助手,正在回复群成员的消息。
要求:
- 用简洁的中文回答(除非用户用其他语言提问)
- 使用企业微信支持的 Markdown 子集(加粗、链接、列表;避免复杂表格)
- 直接给出结论,不要冗长铺垫
- 若任务涉及 skills.sh可说明安装命令 `npx skills add owner/repo/skill-name`
- **不要**在回复里写 `[图片]` 占位符;企微无法通过 Markdown 显示图片
- 若用户要页面截图,请明确告知其发送:`截图` 或 `preview`(由 bot 自动发图)
用户任务:
"""
def _cursor_settings() -> dict[str, str | int]:
timeout_raw = env_config.env("CURSOR_TIMEOUT", "600") or "600"
return {
"api_key": env_config.env("CURSOR_API_KEY"),
"cwd": env_config.env("CURSOR_CWD", r"d:\LY\test\tech") or r"d:\LY\test\tech",
"model": env_config.env("CURSOR_MODEL", "composer-2.5") or "composer-2.5",
"timeout": int(timeout_raw),
}
def strip_mention(text: str) -> str:
return re.sub(r"@\S+\s*", "", text).strip()
def _build_prompt(task: str) -> str:
return WECHAT_SYSTEM_PREFIX + task.strip()
def execute_cursor_task_sync(task: str) -> str:
from cursor_sdk import Agent, AgentOptions, CursorAgentError, LocalAgentOptions
settings = _cursor_settings()
api_key = settings["api_key"]
if not api_key:
raise RuntimeError(
"未配置 CURSOR_API_KEY。请在 bot/.env 中设置,"
"密钥见 https://cursor.com/dashboard/integrations"
)
warm_cursor_bridge()
cwd = str(settings["cwd"])
prompt = _build_prompt(task)
logger.info("Cursor 执行任务 cwd=%s model=%s", cwd, settings["model"])
try:
result = Agent.prompt(
prompt,
AgentOptions(
api_key=api_key,
model=settings["model"],
local=LocalAgentOptions(cwd=cwd),
),
)
except CursorAgentError as exc:
raise RuntimeError(
f"Cursor 启动失败:{exc.message}"
+ ("(可重试)" if exc.is_retryable else "")
) from exc
if result.status == "error":
detail = result.result or "运行失败,无详细错误"
raise RuntimeError(f"Cursor 执行失败:{detail}")
text = (result.result or "").strip()
if not text:
return "Cursor 已完成任务,但没有返回文本内容。"
return text
async def run_cursor_task(
task: str,
on_progress: Callable[[str], Awaitable[None]] | None = None,
) -> str:
timeout = int(_cursor_settings()["timeout"])
if on_progress:
await on_progress("Cursor 正在执行任务,请稍候…")
async with _cursor_lock:
try:
return await asyncio.wait_for(
asyncio.to_thread(execute_cursor_task_sync, task),
timeout=timeout,
)
except asyncio.TimeoutError as exc:
raise RuntimeError(f"Cursor 执行超时(>{timeout}s") from exc

16
bot/env_config.py Normal file
View File

@@ -0,0 +1,16 @@
"""加载 bot/.env供各模块在 import 时统一读取环境变量。"""
from __future__ import annotations
import os
from pathlib import Path
from dotenv import load_dotenv
_BOT_DIR = Path(__file__).resolve().parent
load_dotenv(_BOT_DIR / ".env")
load_dotenv(_BOT_DIR / ".env.local", override=True)
def env(key: str, default: str | None = None) -> str | None:
return os.getenv(key, default)

58
bot/image_extract.py Normal file
View File

@@ -0,0 +1,58 @@
"""从文本/Cursor 回复中解析本地截图路径。"""
from __future__ import annotations
import re
from pathlib import Path
import env_config
IMAGE_SUFFIXES = (".png", ".jpg", ".jpeg", ".webp")
def _project_cwd() -> Path:
raw = env_config.env("CURSOR_CWD", r"d:\LY\test\tech") or r"d:\LY\test\tech"
return Path(raw).resolve()
def _resolve_candidate(raw: str, cwd: Path) -> Path | None:
cleaned = raw.strip().strip("`\"'[]()")
if not cleaned or cleaned.startswith("http"):
return None
path = Path(cleaned)
if not path.is_absolute():
path = cwd / path
try:
resolved = path.resolve()
except OSError:
return None
if resolved.is_file() and resolved.suffix.lower() in IMAGE_SUFFIXES:
return resolved
return None
def find_image_paths(text: str) -> list[Path]:
cwd = _project_cwd()
seen: set[Path] = set()
found: list[Path] = []
patterns = [
r"(?:保存(?:至|到)|saved\s+to|screenshot\s*[:])\s*([^\s\n\]]+\.(?:png|jpe?g|webp))",
r"([A-Za-z]:\\[^\s\n\]]+\.(?:png|jpe?g|webp))",
r"([^\s\n\]]+\.(?:png|jpe?g|webp))",
]
for pattern in patterns:
for match in re.finditer(pattern, text, re.IGNORECASE):
path = _resolve_candidate(match.group(1), cwd)
if path and path not in seen:
seen.add(path)
found.append(path)
return found
def strip_fake_image_markdown(text: str) -> str:
text = re.sub(r"^\s*\[图片\]\s*$", "", text, flags=re.MULTILINE)
text = re.sub(r"\n{3,}", "\n\n", text)
return text.strip()

156
bot/main.py Normal file
View File

@@ -0,0 +1,156 @@
"""企业微信智能机器人 · Skills 助手skills 快查 + 截图预览 + Cursor 执行任务)。"""
from __future__ import annotations
import logging
import sys
import env_config
from bridge_manager import shutdown_cursor_bridge, warm_cursor_bridge
from router import route_message, routing_mode
from skills_service import handle_command, warm_feed_cache
from wecom_media import reply_image, upload_image
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s [%(levelname)s] %(message)s",
)
logger = logging.getLogger("skills-bot")
BOT_ID = env_config.env("WECOM_BOT_ID") or env_config.env("WECHAT_BOT_ID")
BOT_SECRET = env_config.env("WECOM_BOT_SECRET") or env_config.env("WECHAT_BOT_SECRET")
def _require_credentials() -> None:
if not BOT_ID or not BOT_SECRET:
print(
"请设置环境变量 WECOM_BOT_ID 和 WECOM_BOT_SECRET\n"
"(企业微信 → 智能机器人 → API 模式 → 长连接)",
file=sys.stderr,
)
sys.exit(1)
def create_client():
from aibot import WSClient, WSClientOptions, generate_req_id
ws_client = WSClient(
WSClientOptions(
bot_id=BOT_ID,
secret=BOT_SECRET,
)
)
@ws_client.on("authenticated")
def on_authenticated():
logger.info("企业微信长连接认证成功,路由模式=%s", routing_mode())
cursor_key = env_config.env("CURSOR_API_KEY")
if cursor_key:
logger.info("CURSOR_API_KEY 已加载(%s…)", cursor_key[:8])
try:
warm_cursor_bridge()
logger.info("Cursor bridge 预启动完成")
except Exception as exc:
logger.warning("Cursor bridge 预启动失败Cursor 任务时会重试): %s", exc)
else:
logger.warning("CURSOR_API_KEY 未配置Cursor 任务将失败")
try:
warm_feed_cache()
logger.info("skills 数据预加载完成")
except Exception as exc:
logger.warning("skills 数据预加载失败: %s", exc)
@ws_client.on("event.enter_chat")
async def on_enter_chat(frame):
help_text = handle_command("help")
extra = (
"\n\n---\n"
"**单页截图**`preview` / `截图` / `预览 [路径或URL]`\n"
"**网页操作**:自然语言多步操作,或 `browser 场景名`\n"
"例:`访问登录页,输入账号密码,点击登录,点击智能体管理,截图`\n"
"场景文件:`bot/scenarios/*.yaml`(可用 `browser xiaobao-agent-manage`"
)
await ws_client.reply_welcome(
frame,
{
"msgtype": "markdown",
"markdown": {"content": help_text + extra},
},
)
@ws_client.on("message.text")
async def on_text(frame):
body = frame.get("body", {})
content = body.get("text", {}).get("content", "")
logger.info("收到消息: %s", content)
stream_id = generate_req_id("stream")
last_progress = ""
async def on_progress(message: str) -> None:
nonlocal last_progress
if message != last_progress:
last_progress = message
await ws_client.reply_stream(frame, stream_id, message, False)
await ws_client.reply_stream(frame, stream_id, "收到,正在处理…", False)
try:
result = await route_message(content, on_progress=on_progress)
reply = result.text
logger.info(
"回复来源: %s, 文本长度=%d, 图片=%s",
result.source,
len(reply),
result.image_path or "-",
)
except Exception as exc:
logger.exception("处理失败")
reply = f"处理失败:{exc}"
result = None
if len(reply) > 3800:
reply = reply[:3800] + "\n\n> …内容已截断"
await ws_client.reply_stream(frame, stream_id, reply, True)
if result and result.image_path:
try:
media_id = await upload_image(ws_client, result.image_path)
await reply_image(ws_client, frame, media_id)
logger.info("图片已发送到企微: %s", result.image_path)
except Exception as exc:
logger.exception("发送图片失败")
await ws_client.reply(
frame,
{
"msgtype": "markdown",
"markdown": {
"content": f"截图文件:`{result.image_path}`\n发图失败:{exc}\n\n请确认 bot 已重启,或发送 `截图` 重试。",
},
},
)
@ws_client.on("error")
def on_error(error):
logger.error("连接错误: %s", error)
@ws_client.on("disconnected")
def on_disconnected(reason):
logger.warning("连接断开: %s", reason)
return ws_client
def main() -> None:
import atexit
atexit.register(shutdown_cursor_bridge)
_require_credentials()
client = create_client()
logger.info("启动 Skills 助手Bot ID=%s", BOT_ID[:8] if BOT_ID else "?")
client.run()
if __name__ == "__main__":
main()

255
bot/preview_service.py Normal file
View File

@@ -0,0 +1,255 @@
"""在 CURSOR_CWD 启动/访问前端并截图(单页,不含多步操作)。"""
from __future__ import annotations
import asyncio
import json
import logging
import re
import socket
import subprocess
import time
from dataclasses import dataclass
from datetime import datetime
from pathlib import Path
from urllib.parse import urlparse
import env_config
logger = logging.getLogger(__name__)
SCREENSHOT_DIR = Path(__file__).resolve().parent / ".cache" / "screenshots"
@dataclass
class PreviewResult:
url: str
screenshot_path: Path
started_dev_server: bool
final_url: str | None = None
@dataclass
class PreviewRequest:
url: str | None
port: int | None
def _project_cwd() -> Path:
raw = env_config.env("CURSOR_CWD", r"d:\LY\test\tech") or r"d:\LY\test\tech"
return Path(raw).resolve()
def _preview_port() -> int:
raw = env_config.env("PREVIEW_PORT", "5173") or "5173"
return int(raw)
def _startup_timeout() -> int:
raw = env_config.env("PREVIEW_STARTUP_TIMEOUT", "120") or "120"
return int(raw)
def _dev_command() -> str:
return env_config.env("PREVIEW_DEV_COMMAND", "npm run dev") or "npm run dev"
def parse_preview_command(text: str) -> tuple[str | None, int | None] | None:
raw = re.sub(r"@\S+\s*", "", text).strip()
if not raw:
return None
m = re.match(
r"^(preview|截图|预览|截屏)(?:\s+(https?://\S+|/\S*))?(?:\s+(\d{2,5}))?$",
raw,
re.IGNORECASE,
)
if not m:
return None
url_part = m.group(2)
port_part = m.group(3)
port = int(port_part) if port_part else None
if url_part and url_part.startswith("/"):
port = port or _preview_port()
return f"http://127.0.0.1:{port}{url_part}", port
return url_part, port
def resolve_preview_request(text: str) -> PreviewRequest | None:
explicit = parse_preview_command(text)
if explicit is not None:
url_override, port_override = explicit
return PreviewRequest(url=url_override, port=port_override)
if not is_preview_intent(text):
return None
url_override = extract_url_from_text(text)
if not url_override:
env_url = env_config.env("PREVIEW_URL")
url_override = env_url.strip() if env_url else f"http://127.0.0.1:{_preview_port()}/"
return PreviewRequest(url=url_override, port=None)
_PREVIEW_INTENT = re.compile(
r"^(preview|截图|预览|截屏)\b|"
r"(页面预览|运行.*(前端|项目|页面)|"
r"打开.*(前端|页面|项目)|"
r"访问.*(并)?.*(截图|截屏)|"
r"启动.*(前端|项目|dev|服务).*(截图|截屏)?)",
re.IGNORECASE,
)
def is_preview_intent(text: str) -> bool:
raw = re.sub(r"@\S+\s*", "", text).strip()
if parse_preview_command(text) is not None:
return True
return bool(_PREVIEW_INTENT.search(raw))
def extract_url_from_text(text: str) -> str | None:
raw = re.sub(r"@\S+\s*", "", text)
match = re.search(
r"(https?://[^\s\]`\"']+|localhost:\d+[/\w\-./]*)",
raw,
re.IGNORECASE,
)
if not match:
return None
url = match.group(1).rstrip(".,,。")
if url.lower().startswith("localhost"):
url = "http://" + url
return url
def _capture_screenshot_sync(url: str, output: Path) -> str:
from playwright.sync_api import sync_playwright
output.parent.mkdir(parents=True, exist_ok=True)
with sync_playwright() as p:
browser = p.chromium.launch(headless=True)
page = browser.new_page(viewport={"width": 1280, "height": 720})
page.goto(url, wait_until="networkidle", timeout=60_000)
page.wait_for_timeout(1500)
page.screenshot(path=str(output), full_page=False, type="png")
final_url = page.url
browser.close()
return final_url
def _wait_for_port(host: str, port: int, timeout: int) -> bool:
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
try:
with socket.create_connection((host, port), timeout=2):
return True
except OSError:
time.sleep(1)
return False
def _resolve_target_url(url_override: str | None, port_override: int | None) -> tuple[str, str | None]:
if url_override:
parsed = urlparse(url_override)
if parsed.scheme and parsed.netloc:
return url_override, None
raise RuntimeError(f"无效 URL{url_override}")
env_url = env_config.env("PREVIEW_URL")
if env_url:
return env_url.strip(), None
port = port_override or _preview_port()
cwd = _project_cwd()
dev_script = _package_dev_script(cwd)
base = f"http://127.0.0.1:{port}/"
return base, dev_script
def _package_dev_script(cwd: Path) -> str | None:
pkg = cwd / "package.json"
if not pkg.exists():
return None
try:
data = json.loads(pkg.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return None
scripts = data.get("scripts") or {}
for key in ("dev", "preview", "start"):
if scripts.get(key):
cmd = _dev_command()
if key != "dev" and cmd == "npm run dev":
return f"npm run {key}"
return cmd
return None
def _capture_preview_sync(url: str, dev_command: str | None) -> PreviewResult:
cwd = _project_cwd()
parsed = urlparse(url)
host = parsed.hostname or "127.0.0.1"
port = parsed.port or (443 if parsed.scheme == "https" else 80)
dev_proc: subprocess.Popen | None = None
started = False
if dev_command:
if _wait_for_port(host, port, timeout=3):
logger.info("检测到端口 %s 已监听,跳过启动 dev server", port)
else:
logger.info("启动 dev server: %s (cwd=%s)", dev_command, cwd)
dev_proc = subprocess.Popen(
dev_command,
cwd=str(cwd),
shell=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
)
started = True
if not _wait_for_port(host, port, timeout=_startup_timeout()):
err = ""
if dev_proc.stderr:
err = dev_proc.stderr.read().decode("utf-8", errors="replace")[-1000:]
raise RuntimeError(
f"dev server 在 {_startup_timeout()}s 内未就绪 ({url})。"
f"{(' 日志: ' + err) if err else ''}"
)
else:
if not _wait_for_port(host, port, timeout=5):
raise RuntimeError(
f"无法访问 {url}。请在 CURSOR_CWD 放置前端项目,"
"或先手动启动 dev server或设置 PREVIEW_URL。"
)
stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
output = SCREENSHOT_DIR / f"preview-{stamp}.png"
try:
final_url = _capture_screenshot_sync(url, output)
finally:
if dev_proc and dev_proc.poll() is None:
dev_proc.terminate()
try:
dev_proc.wait(timeout=5)
except subprocess.TimeoutExpired:
dev_proc.kill()
return PreviewResult(
url=url,
screenshot_path=output,
started_dev_server=started,
final_url=final_url,
)
async def capture_preview(
url_override: str | None = None,
port_override: int | None = None,
) -> PreviewResult:
url, dev_command = _resolve_target_url(url_override, port_override)
return await asyncio.to_thread(_capture_preview_sync, url, dev_command)

7
bot/requirements.txt Normal file
View File

@@ -0,0 +1,7 @@
wecom-aibot-python-sdk>=1.0.2
python-dotenv>=1.0.0
httpx>=0.27.0
certifi>=2024.0.0
cursor-sdk>=0.1.0
playwright>=1.49.0
PyYAML>=6.0.0

109
bot/router.py Normal file
View File

@@ -0,0 +1,109 @@
"""消息路由skills 快查 / 网页操作 / 截图预览 / Cursor 通用任务。"""
from __future__ import annotations
import re
import env_config
from browser_parser import is_browser_intent, parse_browser_request
from browser_service import format_browser_caption, run_browser_automation
from cursor_runner import run_cursor_task, strip_mention
from image_extract import find_image_paths, strip_fake_image_markdown
from preview_service import capture_preview, is_preview_intent, resolve_preview_request
from skills_service import handle_command, parse_command
from bot_types import RouteResult
def routing_mode() -> str:
return (env_config.env("ROUTING_MODE", "hybrid") or "hybrid").lower()
def _normalize(text: str) -> str:
return re.sub(r"@\S+\s*", "", text).strip().lower()
def is_skills_fast_command(text: str) -> bool:
raw = _normalize(text)
if not raw:
return True
if raw in {"help", "帮助", "?", "h"}:
return True
cmd = parse_command(text)
if cmd.kind in {"help", "list", "detail"}:
return True
if cmd.kind == "search" and re.match(r"^(search|搜索|find|查)\s+", raw):
return True
return False
async def _run_browser(text: str, on_progress=None) -> RouteResult:
if parse_browser_request(text) is None:
raise RuntimeError("无法解析网页操作步骤")
if on_progress:
await on_progress("正在按步骤执行网页操作…")
result = await run_browser_automation(text)
return RouteResult(
source="browser",
text=format_browser_caption(result),
image_path=str(result.screenshot_path),
)
async def _run_preview(text: str, on_progress=None) -> RouteResult:
preview_req = resolve_preview_request(text)
if preview_req is None:
raise RuntimeError("无法解析截图请求")
if on_progress:
await on_progress(f"正在访问并截图:{preview_req.url or '默认地址'}")
result = await capture_preview(preview_req.url, preview_req.port)
caption = (
f"**页面预览**\n"
f"> URL`{result.final_url or result.url}`\n"
f"> 项目:`{env_config.env('CURSOR_CWD', '')}`\n"
f"> dev server{'已自动启动' if result.started_dev_server else '使用已有服务'}"
)
return RouteResult(
source="preview",
text=caption,
image_path=str(result.screenshot_path),
)
async def route_message(text: str, on_progress=None) -> RouteResult:
task = strip_mention(text)
if not task:
return RouteResult("skills", handle_command("help"))
if is_browser_intent(text):
return await _run_browser(text, on_progress=on_progress)
if resolve_preview_request(text) is not None:
return await _run_preview(text, on_progress=on_progress)
mode = routing_mode()
if mode == "skills":
return RouteResult("skills", handle_command(text))
if mode == "cursor" or not is_skills_fast_command(text):
reply = await run_cursor_task(task, on_progress=on_progress)
reply = strip_fake_image_markdown(reply)
image_path: str | None = None
paths = find_image_paths(reply)
if paths:
image_path = str(paths[0])
elif is_preview_intent(text) or is_browser_intent(text):
if on_progress:
await on_progress("未找到截图文件,改用 Playwright 自动执行…")
if is_browser_intent(text):
return await _run_browser(text, on_progress=on_progress)
return await _run_preview(text, on_progress=on_progress)
return RouteResult("cursor", reply, image_path=image_path)
return RouteResult("skills", handle_command(text))

View File

@@ -0,0 +1,17 @@
name: xiaobao-agent-manage
description: 登录后打开智能体管理并截图
steps:
- goto: /login
- fill:
field: 账号
value: "{{PREVIEW_LOGIN_USER}}"
- fill:
field: 密码
value: "{{PREVIEW_LOGIN_PASSWORD}}"
- click: 登录
- wait:
url: "**/app/**"
timeout: 60000
- click: 智能体管理
- wait: 1500
- screenshot

317
bot/skills_service.py Normal file
View File

@@ -0,0 +1,317 @@
"""skills.sh 数据查询与命令解析。"""
from __future__ import annotations
import json
import logging
import re
import time
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Literal
import certifi
import httpx
logger = logging.getLogger(__name__)
FEED_URLS = [
# jsDelivr 在国内通常比 raw.githubusercontent.com 更稳定
"https://cdn.jsdelivr.net/gh/NeverSight/skills.sh_feed@main/data/feed.json",
"https://raw.githubusercontent.com/NeverSight/skills.sh_feed/main/data/feed.json",
]
CACHE_TTL_SECONDS = 600
CACHE_DIR = Path(__file__).resolve().parent / ".cache"
CACHE_FILE = CACHE_DIR / "feed.json"
_cache: dict[str, Any] = {"data": None, "fetched_at": 0.0}
Board = Literal["trending", "hot", "all"]
@dataclass
class Command:
kind: Literal["help", "list", "search", "detail"]
board: Board = "trending"
limit: int = 10
query: str = ""
def _fetch_json(url: str) -> dict[str, Any]:
headers = {
"User-Agent": "skills-hot-bot/1.0",
"Accept": "application/json",
}
with httpx.Client(
timeout=httpx.Timeout(20.0, connect=10.0),
verify=certifi.where(),
follow_redirects=True,
) as client:
resp = client.get(url, headers=headers)
resp.raise_for_status()
return resp.json()
def _load_disk_cache() -> dict[str, Any] | None:
if not CACHE_FILE.exists():
return None
try:
return json.loads(CACHE_FILE.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError) as exc:
logger.warning("读取本地缓存失败: %s", exc)
return None
def _save_disk_cache(data: dict[str, Any]) -> None:
CACHE_DIR.mkdir(parents=True, exist_ok=True)
CACHE_FILE.write_text(json.dumps(data, ensure_ascii=False), encoding="utf-8")
def load_feed(force: bool = False) -> dict[str, Any]:
now = time.time()
if not force and _cache["data"] and now - _cache["fetched_at"] < CACHE_TTL_SECONDS:
return _cache["data"]
errors: list[str] = []
for url in FEED_URLS:
for attempt in range(3):
try:
data = _fetch_json(url)
_cache["data"] = data
_cache["fetched_at"] = now
_save_disk_cache(data)
logger.info("skills 数据已更新: %s", url)
return data
except Exception as exc:
msg = f"{url} (#{attempt + 1}): {exc}"
errors.append(msg)
logger.debug("拉取失败 %s", msg)
time.sleep(0.5 * (attempt + 1))
stale = _load_disk_cache()
if stale:
logger.warning("网络不可用,回退到本地缓存")
_cache["data"] = stale
_cache["fetched_at"] = now
return stale
raise RuntimeError(f"无法获取 skills 数据。最近错误: {errors[-1]}")
def warm_feed_cache() -> None:
"""启动时预加载,避免首条消息才触发网络请求。"""
load_feed(force=True)
def _normalize_text(text: str) -> str:
text = re.sub(r"@\S+\s*", "", text)
return text.strip().lower()
def _parse_limit(raw: str | None, default: int = 10) -> int:
if not raw:
return default
try:
n = int(raw)
except ValueError:
return default
return max(1, min(n, 30))
def _match_list(raw: str, board: Board, aliases: str) -> Command | None:
m = re.match(rf"^({aliases})(?:\s+top)?\s*(\d+)?$", raw)
if m:
return Command(kind="list", board=board, limit=_parse_limit(m.group(2)))
m = re.match(rf"^(查|查询)\s+({aliases})(?:\s+top)?\s*(\d+)?$", raw)
if m:
return Command(kind="list", board=board, limit=_parse_limit(m.group(3)))
return None
def parse_command(text: str) -> Command:
raw = _normalize_text(text)
if not raw or raw in {"help", "帮助", "?", "h"}:
return Command(kind="help")
for board, aliases in (
("trending", "trending|趋势|top"),
("hot", "hot|实时|热门"),
("all", "all|总榜|alltime|all-time"),
):
cmd = _match_list(raw, board, aliases)
if cmd:
return cmd
m = re.match(r"^(search|搜索|find|查)\s+(.+)$", raw)
if m:
return Command(kind="search", query=m.group(2).strip(), limit=5)
m = re.match(r"^(detail|详情|skill|info)\s+(.+)$", raw)
if m:
return Command(kind="detail", query=m.group(2).strip())
if raw.startswith("trending") or raw.startswith("趋势"):
parts = raw.split(maxsplit=1)
return Command(kind="list", board="trending", limit=_parse_limit(parts[1] if len(parts) > 1 else None))
return Command(kind="search", query=raw, limit=5)
def _format_installs(n: int | float) -> str:
if n >= 1_000_000:
return f"{n / 1_000_000:.1f}M"
if n >= 1_000:
return f"{n / 1_000:.1f}K"
return str(int(n))
def _board_items(feed: dict[str, Any], board: Board) -> list[dict[str, Any]]:
key = {"trending": "topTrending", "hot": "topHot", "all": "topAllTime"}[board]
return feed.get(key, [])
def _board_title(board: Board) -> str:
return {
"trending": "Trending近期增长",
"hot": "Hot实时热度",
"all": "All Time总安装榜",
}[board]
def format_list(board: Board, limit: int) -> str:
feed = load_feed()
items = _board_items(feed, board)[:limit]
updated = feed.get("updatedAt", "未知")[:10]
lines = [
f"**skills.sh {_board_title(board)} Top {limit}**",
f"> 数据更新:{updated}",
"",
]
for i, item in enumerate(items, 1):
title = item.get("title", "?")
source = item.get("source", "?")
installs = _format_installs(item.get("installs", 0))
desc = item.get("description", "")
if len(desc) > 80:
desc = desc[:77] + "..."
link = item.get("link", "")
lines.append(f"{i}. **{title}** · {installs}")
lines.append(f" `{source}`")
if desc:
lines.append(f" {desc}")
if link:
lines.append(f" [查看]({link})")
lines.append("")
return "\n".join(lines).strip()
def format_search(query: str, limit: int) -> str:
feed = load_feed()
q = query.lower()
seen: set[str] = set()
matches: list[dict[str, Any]] = []
for board in ("topTrending", "topHot", "topAllTime"):
for item in feed.get(board, []):
item_id = item.get("id") or item.get("title", "")
if item_id in seen:
continue
haystack = " ".join(
[
item.get("title", ""),
item.get("source", ""),
item.get("description", ""),
]
).lower()
if q in haystack:
seen.add(item_id)
matches.append(item)
if len(matches) >= limit:
break
if len(matches) >= limit:
break
if not matches:
return f"未找到与 **{query}** 相关的 skill。\n\n试试:`trending 10` / `hot 10` / `搜索 react`"
lines = [f"**搜索「{query}」** 共 {len(matches)}", ""]
for i, item in enumerate(matches, 1):
title = item.get("title", "?")
source = item.get("source", "?")
installs = _format_installs(item.get("installs", 0))
link = item.get("link", "")
lines.append(f"{i}. **{title}** · {installs} · `{source}`")
if link:
lines.append(f" [查看]({link})")
return "\n".join(lines)
def format_detail(name: str) -> str:
feed = load_feed()
q = name.lower().strip()
best: dict[str, Any] | None = None
for board in ("topTrending", "topHot", "topAllTime"):
for item in feed.get(board, []):
title = (item.get("title") or "").lower()
item_id = (item.get("id") or "").lower()
if title == q or q in title or q in item_id:
if best is None or item.get("installs", 0) > best.get("installs", 0):
best = item
if not best:
return f"未找到 skill**{name}**\n\n试试:`搜索 {name}`"
desc = best.get("description", "无描述")
return "\n".join(
[
f"**{best.get('title', '?')}**",
f"`{best.get('source', '?')}`",
f"安装量:**{_format_installs(best.get('installs', 0))}**",
"",
desc,
"",
f"[skills.sh 详情]({best.get('link', 'https://skills.sh')})",
"",
f"安装:`npx skills add {best.get('source', '')}/{best.get('title', '')}`",
]
)
def format_help() -> str:
return "\n".join(
[
"**Skills 助手 · 命令帮助**",
"",
"`trending 10` / `趋势 10` — 近期增长榜",
"`hot 10` / `实时 10` — 实时热度榜",
"`all 10` / `总榜 10` — 历史总安装榜",
"`搜索 react` / `search tdd` — 关键词搜索",
"`详情 find-skills` — 查看单个 skill",
"`preview` / `截图` / `预览` — 单页截图",
"`browser 场景名` — 执行 YAML 场景(见 bot/scenarios/",
"自然语言 — 如:访问登录页,输入账号密码,点击登录,点击智能体管理,截图",
"`preview /about 5173` — 指定路径和端口",
"",
"示例:",
"• trending top10",
"• 查 grill",
"• 详情 remotion-render",
]
)
def handle_command(text: str) -> str:
cmd = parse_command(text)
if cmd.kind == "help":
return format_help()
if cmd.kind == "list":
return format_list(cmd.board, cmd.limit)
if cmd.kind == "search":
return format_search(cmd.query, cmd.limit)
if cmd.kind == "detail":
return format_detail(cmd.query)
return format_help()

96
bot/wecom_media.py Normal file
View File

@@ -0,0 +1,96 @@
"""企业微信 API 模式:上传图片并回复。"""
from __future__ import annotations
import base64
import hashlib
import logging
from pathlib import Path
from typing import Any
from aibot import generate_req_id
logger = logging.getLogger(__name__)
CHUNK_SIZE = 512 * 1024
MAX_IMAGE_BYTES = 9 * 1024 * 1024
def _ensure_image_size(path: Path) -> bytes:
data = path.read_bytes()
if len(data) > MAX_IMAGE_BYTES:
raise RuntimeError(
f"截图过大({len(data) // 1024}KB请缩小页面或使用 viewport 截图(上限 9MB"
)
return data
def _response_body(frame: dict[str, Any]) -> dict[str, Any]:
if frame.get("errcode", 0) != 0:
raise RuntimeError(
f"企微接口错误 errcode={frame.get('errcode')} errmsg={frame.get('errmsg')}"
)
body = frame.get("body")
return body if isinstance(body, dict) else {}
async def upload_image(ws_client: Any, image_path: str | Path) -> str:
path = Path(image_path)
if not path.exists():
raise RuntimeError(f"截图不存在: {path}")
data = _ensure_image_size(path)
md5 = hashlib.md5(data).hexdigest()
chunks = [data[i : i + CHUNK_SIZE] for i in range(0, len(data), CHUNK_SIZE)]
total_chunks = len(chunks)
manager = ws_client._ws_manager
init_frame = await manager.send_reply(
generate_req_id("upload_init"),
{
"type": "image",
"filename": path.name,
"total_size": len(data),
"total_chunks": total_chunks,
"md5": md5,
},
"aibot_upload_media_init",
)
upload_id = _response_body(init_frame).get("upload_id")
if not upload_id:
raise RuntimeError("上传初始化失败:未返回 upload_id")
for index, chunk in enumerate(chunks):
chunk_frame = await manager.send_reply(
generate_req_id("upload_chunk"),
{
"upload_id": upload_id,
"chunk_index": index,
"base64_data": base64.b64encode(chunk).decode("ascii"),
},
"aibot_upload_media_chunk",
)
_response_body(chunk_frame)
finish_frame = await manager.send_reply(
generate_req_id("upload_finish"),
{"upload_id": upload_id},
"aibot_upload_media_finish",
)
media_id = _response_body(finish_frame).get("media_id")
if not media_id:
raise RuntimeError("上传完成但未返回 media_id")
logger.info("图片已上传 media_id=%s", str(media_id)[:12])
return str(media_id)
async def reply_image(ws_client: Any, frame: dict[str, Any], media_id: str) -> None:
await ws_client.reply(
frame,
{
"msgtype": "image",
"image": {"media_id": media_id},
},
)

6
daily/__init__.py Normal file
View File

@@ -0,0 +1,6 @@
"""早报生成与企微推送。"""
from daily.generate import generate_report, main as generate_main
from daily.webhook import send_report
__all__ = ["generate_report", "generate_main", "send_report"]

22
daily/__main__.py Normal file
View File

@@ -0,0 +1,22 @@
"""CLI: python -m daily [generate|push] [report_path]"""
from __future__ import annotations
import sys
from daily.generate import main as generate_main
from daily.webhook import main as push_main
def main() -> int:
cmd = (sys.argv[1] if len(sys.argv) > 1 else "generate").lower()
if cmd in {"generate", "gen", "g"}:
return generate_main()
if cmd in {"push", "send", "webhook"}:
return push_main(sys.argv[2:])
print(f"未知命令: {cmd}\n用法: python -m daily [generate|push] [report_path]", file=sys.stderr)
return 1
if __name__ == "__main__":
raise SystemExit(main())

134
daily/agent_workflow.py Normal file
View File

@@ -0,0 +1,134 @@
"""Agent 工作流:趋势分析 → 叙事化企微早报。"""
from __future__ import annotations
import json
import logging
import re
from pathlib import Path
from typing import Any
from daily.config import OUTPUT_DIR, ROOT, env
from daily.llm_client import extract_json_object, has_llm_configured, llm_chat
from daily.report_data import save_json
logger = logging.getLogger(__name__)
_SKILL_DIR = ROOT / "skills" / "daily-agent"
_MD_BLOCK = re.compile(r"```(?:markdown|md)?\s*([\s\S]*?)```", re.IGNORECASE)
_WECOM_NEW_ENTRY_NOTE = re.compile(r"(新入[^]*")
def _strip_new_entry_notes(md: str) -> str:
md = _WECOM_NEW_ENTRY_NOTE.sub("", md)
return re.sub(r"\*\*—", "** —", md)
def report_mode() -> str:
return (env("DAILY_REPORT_MODE") or "classic").strip().lower()
def is_agent_mode() -> bool:
if report_mode() != "agent":
return False
if not has_llm_configured():
logger.warning("DAILY_REPORT_MODE=agent 但未配置 LLM回退 classic")
return False
return True
def trends_json_path(date_str: str) -> Path:
return OUTPUT_DIR / f"{date_str}.trends.json"
def _load_skill() -> str:
path = _SKILL_DIR / "SKILL.md"
if path.exists():
return path.read_text(encoding="utf-8").strip()
return "你是早报主编 Agent。"
def _extract_markdown(text: str) -> str:
text = text.strip()
match = _MD_BLOCK.search(text)
if match:
return match.group(1).strip()
if text.startswith("📰"):
return text
return text
def analyze_trends(llm_input: dict[str, Any], *, date_str: str) -> dict[str, Any] | None:
skill = _load_skill()
system = (
f"{skill}\n\n"
"当前执行 **Step 1趋势分析**。\n"
"只输出 trends JSONheadline, opening, themes, top_picks, signals不要 Markdown。"
)
user = json.dumps(llm_input, ensure_ascii=False, indent=2)
try:
raw = llm_chat(system, user)
except Exception as exc:
logger.warning("Agent Step1 趋势分析失败:%s", exc)
return None
if not raw:
return None
parsed = extract_json_object(raw)
if not parsed.get("headline") and not parsed.get("opening"):
logger.warning("Agent Step1 JSON 无效")
return None
save_json(trends_json_path(date_str), parsed)
logger.info("Agent Step1 完成:%s", parsed.get("headline", "?"))
return parsed
def write_wecom_report(
llm_input: dict[str, Any],
trends: dict[str, Any],
*,
date_str: str,
time_str: str,
updated: str,
) -> str | None:
skill = _load_skill()
system = (
f"{skill}\n\n"
"当前执行 **Step 2撰写企微早报**。\n"
f"日期={date_str},时间={time_str},数据截至={updated}\n"
"只输出企微 Markdown 正文,不要代码块,不要 JSON。"
)
payload = {"data": llm_input, "trends": trends}
user = json.dumps(payload, ensure_ascii=False, indent=2)
try:
raw = llm_chat(system, user)
except Exception as exc:
logger.warning("Agent Step2 写稿失败:%s", exc)
return None
if not raw:
return None
md = _extract_markdown(raw)
if not md.startswith("📰"):
md = f"📰 **早报 · {date_str}**\n> ⏱ {time_str} · 数据截至 {updated}\n\n{md}"
md = _strip_new_entry_notes(md)
logger.info("Agent Step2 完成:%d bytes", len(md.encode("utf-8")))
return md
def run_agent_workflow(
llm_input: dict[str, Any],
*,
date_str: str,
time_str: str,
updated: str,
) -> str | None:
"""两步 Agent 工作流;成功返回企微 Markdown失败返回 None。"""
trends = analyze_trends(llm_input, date_str=date_str)
if not trends:
return None
return write_wecom_report(
llm_input,
trends,
date_str=date_str,
time_str=time_str,
updated=updated,
)

81
daily/config.py Normal file
View File

@@ -0,0 +1,81 @@
"""项目路径与环境变量加载。"""
from __future__ import annotations
import os
import sys
from pathlib import Path
from dotenv import load_dotenv
ROOT = Path(__file__).resolve().parent.parent
BOT_DIR = ROOT / "bot"
OUTPUT_DIR = ROOT / "output"
LOG_DIR = ROOT / "logs"
CACHE_DIR = ROOT / ".cache"
SNAPSHOT_FILE = CACHE_DIR / "last-report.json"
# 企微 webhook markdown.content 硬上限
WECOM_MARKDOWN_LIMIT = 4096
def wecom_chunk_bytes() -> int:
"""单条企微 markdown 消息字节上限。"""
chunk = env_int("DAILY_WECOM_CHUNK_BYTES", -1)
if chunk < 0:
chunk = env_int("DAILY_WECOM_MAX_BYTES", WECOM_MARKDOWN_LIMIT)
return min(chunk, WECOM_MARKDOWN_LIMIT)
def wecom_skill_desc_limit() -> int:
"""企微 Skills 榜单条简介建议字数。"""
return env_int("DAILY_WECOM_SKILL_DESC_LIMIT", 56)
def full_desc_limit() -> int:
"""完整版早报摘要长度0 表示不截断。"""
return env_int("DAILY_FULL_DESC_LIMIT", 0)
def news_summary_limit() -> int:
return env_int("DAILY_FULL_NEWS_SUMMARY_LIMIT", 0)
def wecom_max_bytes() -> int:
"""兼容旧配置名。"""
return wecom_chunk_bytes()
load_dotenv(ROOT / ".env")
load_dotenv(ROOT / ".env.local", override=True)
def ensure_bot_on_path() -> None:
bot = str(BOT_DIR)
if bot not in sys.path:
sys.path.insert(0, bot)
def _clean_env_value(raw: str | None) -> str | None:
if raw is None:
return None
value = raw.strip()
if not value:
return None
# .env 行内注释(未加引号时 python-dotenv 不会自动去掉)
if " #" in value:
value = value.split(" #", 1)[0].rstrip()
return value or None
def env(key: str, default: str | None = None) -> str | None:
return _clean_env_value(os.getenv(key, default))
def env_int(key: str, default: int) -> int:
raw = env(key)
if not raw:
return default
try:
return int(raw)
except ValueError:
return default

144
daily/cursor_editor.py Normal file
View File

@@ -0,0 +1,144 @@
"""Cursor 编辑层JSON 数据 → 主题 / 速览 / 中文描述。"""
from __future__ import annotations
import json
import logging
from pathlib import Path
from typing import Any
from daily.config import ROOT, env
from daily.llm_client import extract_json_object, has_llm_configured, llm_chat
from daily.text_utils import clip_text
from daily.report_data import editorial_json_path, save_json, skill_id
logger = logging.getLogger(__name__)
_SKILL_DIR = ROOT / "skills" / "daily-editor"
def is_enabled() -> bool:
raw = (env("DAILY_CURSOR_EDITOR") or "").strip().lower()
if raw in {"1", "true", "yes", "on"}:
return has_llm_configured()
if raw in {"0", "false", "no", "off"}:
return False
return False
def _load_skill_prompt() -> str:
skill_path = _SKILL_DIR / "SKILL.md"
if skill_path.exists():
return skill_path.read_text(encoding="utf-8").strip()
return "你是技术早报编辑。根据输入 JSON 输出编辑结果 JSON。"
def _build_system_prompt() -> str:
skill = _load_skill_prompt()
return (
f"{skill}\n\n"
"再次强调:只输出 JSON 对象,包含 theme_line、highlights3条、descriptions。"
)
def run_editorial(llm_input: dict[str, Any], *, date_str: str) -> dict[str, Any] | None:
"""调用 LLM 生成 editorial失败返回 None。"""
if not is_enabled():
return None
system = _build_system_prompt()
user = json.dumps(llm_input, ensure_ascii=False, indent=2)
try:
raw = llm_chat(system, user)
except Exception as exc:
logger.warning("Cursor 编辑失败,回退规则模式:%s", exc)
return None
if not raw:
logger.warning("Cursor 编辑无响应,回退规则模式")
return None
parsed = extract_json_object(raw)
if not parsed.get("theme_line") and not parsed.get("descriptions"):
logger.warning("Cursor 编辑 JSON 无效,回退规则模式")
return None
editorial = _normalize_editorial(parsed)
save_json(editorial_json_path(date_str), editorial)
return editorial
def _normalize_editorial(raw: dict[str, Any]) -> dict[str, Any]:
theme = str(raw.get("theme_line") or "").strip()
highlights_raw = raw.get("highlights") or []
highlights: list[str] = []
if isinstance(highlights_raw, list):
for item in highlights_raw:
if isinstance(item, str) and item.strip():
highlights.append(item.strip())
descriptions_raw = raw.get("descriptions") or {}
descriptions: dict[str, str] = {}
if isinstance(descriptions_raw, dict):
for key, value in descriptions_raw.items():
if isinstance(value, str) and value.strip():
limit = 40 if str(key).startswith("github:") else 36
descriptions[str(key)] = clip_text(value, limit)
return {
"theme_line": theme,
"highlights": highlights[:3],
"descriptions": descriptions,
}
def theme_line_from_editorial(editorial: dict[str, Any]) -> str:
theme = editorial.get("theme_line", "")
if not theme:
return ""
if "今日主题" in theme:
return theme if theme.startswith("**") else f"**{theme}**"
return f"**今日主题**{theme}"
def apply_descriptions(
*,
trending: list[dict[str, Any]],
hot: list[dict[str, Any]],
github_trending: list[dict[str, Any]],
github_emerging: list[dict[str, Any]],
github_topic: list[dict[str, Any]],
ai_news: dict[str, Any],
descriptions: dict[str, str],
) -> None:
if not descriptions:
return
for item in trending + hot:
key = f"skill:{skill_id(item)}"
if key in descriptions:
item["description"] = descriptions[key]
for repo_list in (github_trending, github_emerging, github_topic):
for item in repo_list:
key = f"github:{item.get('repo', '')}"
if key in descriptions:
item["description"] = descriptions[key]
if not ai_news.get("enabled"):
return
def _apply_news_item(item: dict[str, Any]) -> None:
key = f"news:{item.get('link', '')}"
if key in descriptions:
item["summary"] = descriptions[key]
for cat in ai_news.get("categories") or []:
for item in cat.get("items") or []:
_apply_news_item(item)
for item in ai_news.get("flat") or []:
_apply_news_item(item)
def load_cached_editorial(date_str: str) -> dict[str, Any] | None:
path = editorial_json_path(date_str)
if not path.exists():
return None
try:
return _normalize_editorial(json.loads(path.read_text(encoding="utf-8")))
except (OSError, json.JSONDecodeError):
return None

313
daily/delta.py Normal file
View File

@@ -0,0 +1,313 @@
"""榜单异动:对比昨日 Top N仅识别「新入榜」条目分榜、限条"""
from __future__ import annotations
import json
import logging
from datetime import datetime, timedelta
from pathlib import Path
from typing import Any, Callable
from daily.config import OUTPUT_DIR, env_int
logger = logging.getLogger(__name__)
KeyFn = Callable[[dict[str, Any]], str]
def compare_depth() -> int:
return env_int("DAILY_DELTA_COMPARE_DEPTH", 15)
def wecom_new_limit() -> int:
return env_int("DAILY_WECOM_NEW_MAX", 10)
def skill_id(item: dict[str, Any]) -> str:
return str(item.get("id") or f"{item.get('source')}/{item.get('title')}")
def _data_json_path(date_str: str) -> Path:
return OUTPUT_DIR / f"{date_str}.data.json"
def _load_data_json_file(path: Path) -> dict[str, Any]:
return json.loads(path.read_text(encoding="utf-8"))
def _key_set(items: list[dict[str, Any]], key_fn: KeyFn, *, depth: int) -> set[str]:
return {key_fn(item) for item in items[:depth] if key_fn(item)}
def find_previous_data(date_str: str) -> tuple[str, dict[str, Any]] | None:
"""查找最近一份早于 date_str 的 data.json。"""
try:
dt = datetime.strptime(date_str, "%Y-%m-%d")
except ValueError:
return None
lookback = env_int("DAILY_DELTA_LOOKBACK_DAYS", 7)
for days in range(1, lookback + 1):
prev_date = (dt - timedelta(days=days)).strftime("%Y-%m-%d")
path = _data_json_path(prev_date)
if not path.exists():
continue
try:
payload = _load_data_json_file(path)
except (OSError, ValueError) as exc:
logger.warning("读取异动基准 %s 失败:%s", path, exc)
continue
data = payload.get("data")
if isinstance(data, dict) and data.get("date"):
return prev_date, data
return None
def _prev_board_items(prev_data: dict[str, Any], board: str, depth: int) -> list[dict[str, Any]]:
baseline = prev_data.get("movement_baseline") or {}
if board in baseline and isinstance(baseline[board], list):
return baseline[board][:depth]
legacy = {
"skills_trending": "skills_trending",
"skills_hot": "skills_hot",
"github_trending": "github_trending",
"github_emerging": "github_emerging",
"github_topic": "github_topic",
}
if board == "github_topic":
topic = prev_data.get("github_topic") or {}
repos = topic.get("repos") if isinstance(topic, dict) else []
return (repos or [])[:depth]
field = legacy.get(board, board)
items = prev_data.get(field) or []
return items[:depth] if isinstance(items, list) else []
def _format_new_note(board: str, rank: int, *, topic_name: str = "llm") -> str:
labels = {
"trending": "Skills Trending",
"hot": "Skills Hot",
"github_trending": "GitHub Trending",
"github_emerging": "GitHub 新兴",
"github_topic": f"Topic `{topic_name}`",
}
return f"新入 {labels.get(board, board)} #{rank}"
def _build_skill_board_moves(
*,
board: str,
items: list[dict[str, Any]],
prev_data: dict[str, Any] | None,
depth: int,
) -> list[dict[str, Any]]:
board_key = f"skills_{board}"
prev_ids = (
_key_set(_prev_board_items(prev_data, board_key, depth), skill_id, depth=depth)
if prev_data
else set()
)
moves: list[dict[str, Any]] = []
for rank, item in enumerate(items[:depth], 1):
sid = skill_id(item)
if not sid or not prev_data or sid in prev_ids:
continue
moves.append(
{
"kind": "skill",
"board": board,
"id": sid,
"title": item.get("title", ""),
"source": item.get("source", ""),
"installs": int(item.get("installs") or 0),
"link": item.get("link", ""),
"description": item.get("description", ""),
"rank": rank,
"is_new": True,
"note": _format_new_note(board, rank),
}
)
return moves
def _repo_key(item: dict[str, Any]) -> str:
return str(item.get("repo") or "")
def _build_github_board_moves(
*,
board: str,
items: list[dict[str, Any]],
prev_data: dict[str, Any] | None,
depth: int,
topic_name: str = "llm",
) -> list[dict[str, Any]]:
prev_ids = (
_key_set(_prev_board_items(prev_data, board, depth), _repo_key, depth=depth)
if prev_data
else set()
)
moves: list[dict[str, Any]] = []
for rank, item in enumerate(items[:depth], 1):
repo = _repo_key(item)
if not repo or not prev_data or repo in prev_ids:
continue
moves.append(
{
"kind": "github",
"board": board,
"repo": repo,
"url": item.get("url", ""),
"language": item.get("language", ""),
"stars_today_fmt": item.get("stars_today_fmt", ""),
"total_stars_fmt": item.get("total_stars_fmt", ""),
"created_at": item.get("created_at", ""),
"description": item.get("description", ""),
"rank": rank,
"is_new": True,
"note": _format_new_note(board, rank, topic_name=topic_name),
}
)
return moves
def _board_summary(
*,
label: str,
baseline_date: str | None,
depth: int,
moves: list[dict[str, Any]],
capped: list[dict[str, Any]],
) -> str:
if not baseline_date:
return f"无历史基准,无法判断 {label} Top{depth} 新增"
if not moves:
return f"{baseline_date} Top{depth} 无新增 {label} 条目"
total = len(moves)
shown = len(capped)
if shown < total:
return f"{baseline_date} Top{depth} 新增 {total} 条,企微展示前 {shown}"
return f"{baseline_date} Top{depth} 新增 {total}"
def build_movement_baseline(
*,
trending: list[dict[str, Any]],
hot: list[dict[str, Any]],
github_trending: list[dict[str, Any]],
github_emerging: list[dict[str, Any]],
github_topic: list[dict[str, Any]],
depth: int | None = None,
) -> dict[str, Any]:
n = depth if depth is not None else compare_depth()
return {
"compare_depth": n,
"skills_trending": trending[:n],
"skills_hot": hot[:n],
"github_trending": github_trending[:n],
"github_emerging": github_emerging[:n],
"github_topic": github_topic[:n],
}
def build_movement_context(
*,
date_str: str,
trending: list[dict[str, Any]],
hot: list[dict[str, Any]],
github_trending: list[dict[str, Any]],
github_emerging: list[dict[str, Any]],
github_topic: list[dict[str, Any]],
topic_name: str = "llm",
) -> dict[str, Any]:
"""生成 Agent 可用的新增榜上下文:分 Trending/Hot/Topic企微每榜最多 wecom_new_limit 条。"""
depth = compare_depth()
cap = wecom_new_limit()
baseline = find_previous_data(date_str)
baseline_date = baseline[0] if baseline else None
prev_data = baseline[1] if baseline else None
topic = (topic_name or "llm").strip() or "llm"
skills_trending_all = _build_skill_board_moves(
board="trending", items=trending, prev_data=prev_data, depth=depth
)
skills_hot_all = _build_skill_board_moves(
board="hot", items=hot, prev_data=prev_data, depth=depth
)
github_trending_all = _build_github_board_moves(
board="github_trending", items=github_trending, prev_data=prev_data, depth=depth
)
github_emerging_all = _build_github_board_moves(
board="github_emerging", items=github_emerging, prev_data=prev_data, depth=depth
)
github_topic_all = _build_github_board_moves(
board="github_topic",
items=github_topic,
prev_data=prev_data,
depth=depth,
topic_name=topic,
)
skills_trending_moves = skills_trending_all[:cap]
skills_hot_moves = skills_hot_all[:cap]
github_trending_moves = github_trending_all[:cap]
github_emerging_moves = github_emerging_all[:cap]
github_topic_moves = github_topic_all[:cap]
topic_label = f"Topic `{topic}`"
return {
"baseline_date": baseline_date,
"compare_depth": depth,
"wecom_new_limit": cap,
"selection_mode": "top_n",
"topic_name": topic,
"skills_trending_moves": skills_trending_moves,
"skills_hot_moves": skills_hot_moves,
"skills_trending_stable": not skills_trending_all,
"skills_hot_stable": not skills_hot_all,
"skills_trending_summary": _board_summary(
label="Skills Trending",
baseline_date=baseline_date,
depth=depth,
moves=skills_trending_all,
capped=skills_trending_moves,
),
"skills_hot_summary": _board_summary(
label="Skills Hot",
baseline_date=baseline_date,
depth=depth,
moves=skills_hot_all,
capped=skills_hot_moves,
),
"github_trending_moves": github_trending_moves,
"github_emerging_moves": github_emerging_moves,
"github_topic_moves": github_topic_moves,
"github_trending_stable": not github_trending_all,
"github_emerging_stable": not github_emerging_all,
"github_topic_stable": not github_topic_all,
"github_trending_summary": _board_summary(
label="GitHub Trending",
baseline_date=baseline_date,
depth=depth,
moves=github_trending_all,
capped=github_trending_moves,
),
"github_emerging_summary": _board_summary(
label="GitHub 新兴",
baseline_date=baseline_date,
depth=depth,
moves=github_emerging_all,
capped=github_emerging_moves,
),
"github_topic_summary": _board_summary(
label=topic_label,
baseline_date=baseline_date,
depth=depth,
moves=github_topic_all,
capped=github_topic_moves,
),
# 兼容旧字段(合并,仅供调试)
"skills_moves": skills_trending_moves + skills_hot_moves,
"github_moves": github_trending_moves + github_emerging_moves + github_topic_moves,
"skills_stable": not skills_trending_all and not skills_hot_all,
"github_stable": not github_trending_all and not github_emerging_all and not github_topic_all,
}

281
daily/format_wecom.py Normal file
View File

@@ -0,0 +1,281 @@
"""企微早报排版。"""
from __future__ import annotations
import re
from typing import Any
from daily.config import wecom_skill_desc_limit
from daily.localize import LocalizeJob, localize_brief_descriptions, needs_chinese
from daily.text_utils import trim_brief
ICONS = {
"header": "📰",
"highlights": "💡",
"trending": "📈",
"hot": "🔥",
"github": "🐙",
"emerging": "🌱",
"topic": "🤖",
"ainews": "🌍",
"pick": "📦",
"theme": "🎯",
"file": "📄",
}
def _skill_line(rank: int, item: dict[str, Any], *, badge: str = "") -> list[str]:
source = item.get("source", "?")
installs = item.get("installs_fmt", "?")
link = item.get("link", "")
desc = item.get("desc_short", "")
badge_prefix = f"{badge} " if badge else ""
if item.get("cluster"):
count = int(item.get("cluster_count") or 1)
sample = item.get("cluster_titles") or item.get("title", "")
label = f"**{source}** · {count} skills · **{installs}**"
if link:
head = f"{rank}. {badge_prefix}[{label}]({link})"
else:
head = f"{rank}. {badge_prefix}{label}"
lines = [head]
if sample or desc:
hint = desc or sample
lines.append(f" > {hint}")
return lines
title = item.get("title", "?")
if link:
head = f"{rank}. {badge_prefix}[**{title}**]({link}) · `{source}` · **{installs}**"
else:
head = f"{rank}. {badge_prefix}**{title}** · `{source}` · **{installs}**"
lines = [head]
if desc:
lines.append(f" > {desc}")
return lines
def _ai_news_lines(items: list[dict[str, Any]]) -> list[str]:
lines: list[str] = []
for i, item in enumerate(items, 1):
title = item.get("title", "?")
link = item.get("link", "")
source = item.get("source_name", "?")
pub = item.get("published_fmt", "")
desc = item.get("desc_short", "")
pub_suffix = f" · {pub}" if pub else ""
if link:
head = f"{i}. [**{title}**]({link}) · `{source}`{pub_suffix}"
else:
head = f"{i}. **{title}** · `{source}`{pub_suffix}"
lines.append(head)
if desc:
lines.append(f" > {desc}")
return lines
def _github_repo_lines(repos: list[dict[str, Any]], *, show_created: bool = False) -> list[str]:
lines: list[str] = []
for i, repo in enumerate(repos, 1):
name = repo["repo"]
url = repo["url"]
lang = repo.get("language", "")
stars_today = repo.get("stars_today_fmt", "")
total = repo.get("total_stars_fmt", "")
created = repo.get("created_at", "")
meta_parts: list[str] = []
if lang:
meta_parts.append(lang)
if stars_today:
meta_parts.append(f"+{stars_today} today")
elif total:
meta_parts.append(f"{total}")
if show_created and created:
meta_parts.append(f"创建于 {created}")
meta = f" · {' · '.join(meta_parts)}" if meta_parts else ""
lines.append(f"{i}. [{name}]({url}){meta}")
desc = repo.get("desc_short") or repo.get("description", "")
if desc:
lines.append(f" > {desc}")
return lines
def _fallback_skill_desc(item: dict[str, Any]) -> str:
if item.get("cluster"):
count = int(item.get("cluster_count") or 1)
source = item.get("source") or "unknown"
sample = item.get("cluster_titles") or item.get("title") or ""
return f"{count} agent skills from {source}, including {sample}"
title = item.get("title") or "skill"
source = item.get("source") or "unknown"
return f"{title} skill from {source}"
def _skill_group_key(item: dict[str, Any]) -> str:
return str(item.get("id") or f"{item.get('source')}/{item.get('title')}")
def finalize_wecom_skill_groups(
items: list[dict[str, Any]],
*,
desc_limit: int | None = None,
) -> list[dict[str, Any]]:
"""为企微 Skills 榜生成一句简要中文简介(与完整版 .md 长描述分离)。"""
if desc_limit is None:
desc_limit = wecom_skill_desc_limit()
limit = desc_limit if desc_limit > 0 else 48
copies: list[tuple[str, dict[str, Any]]] = []
jobs: list[LocalizeJob] = []
for item in items:
copy = dict(item)
desc = (copy.get("description") or "").strip()
if not desc:
desc = _fallback_skill_desc(copy)
key = _skill_group_key(copy)
job_key = f"wecom:{key}"
if needs_chinese(desc) or len(desc) > limit:
jobs.append(LocalizeJob(job_key, desc, limit))
else:
copy["wecom_desc"] = desc
copies.append((job_key, copy))
zh_map = localize_brief_descriptions(jobs, archive=True)
out: list[dict[str, Any]] = []
for job_key, copy in copies:
if job_key in zh_map:
copy["wecom_desc"] = zh_map[job_key]
elif "wecom_desc" not in copy:
copy["wecom_desc"] = _brief_fallback_desc(
(copy.get("description") or "").strip() or _fallback_skill_desc(copy),
limit,
)
out.append(copy)
return out
def _brief_fallback_desc(text: str, limit: int) -> str:
return trim_brief(text, limit)
def _grouped_skill_to_wecom_item(
item: dict[str, Any],
*,
desc_limit: int | None = None,
) -> dict[str, Any]:
if desc_limit is None:
desc_limit = wecom_skill_desc_limit()
limit = desc_limit if desc_limit > 0 else 48
installs = int(item.get("installs") or 0)
installs_fmt = item.get("installs_fmt") or str(installs)
desc = (item.get("wecom_desc") or item.get("description") or "").strip()
if not desc and item.get("cluster"):
desc = item.get("cluster_titles") or ""
if not item.get("wecom_desc"):
desc = _brief_fallback_desc(desc, limit)
return {
"title": item.get("title", ""),
"source": item.get("source", "?"),
"installs_fmt": installs_fmt,
"link": item.get("link", ""),
"desc_short": desc,
"cluster": bool(item.get("cluster")),
"cluster_count": item.get("cluster_count"),
"cluster_titles": item.get("cluster_titles"),
}
def build_skills_board_section(icon_key: str, board_label: str, items: list[dict[str, Any]]) -> str:
prepared = finalize_wecom_skill_groups(items)
wecom_items = [_grouped_skill_to_wecom_item(x) for x in prepared]
lines = [f"{ICONS[icon_key]} **{board_label} Top {len(wecom_items)}**"]
for rank, item in enumerate(wecom_items, 1):
lines.extend(_skill_line(rank, item))
return "\n".join(lines)
_SKILL_SECTIONS = re.compile(
r"📈 \*\*Skills Trending.*?(?=🐙 \*\*GitHub Trending)",
re.DOTALL,
)
def replace_wecom_skill_sections(
md: str,
*,
trending: list[dict[str, Any]],
hot: list[dict[str, Any]],
) -> str:
"""用 Python 合并后的 Skills 榜替换或插入 Agent 早报中的对应区块。"""
trending_sec = build_skills_board_section("trending", "Skills Trending", trending)
hot_sec = build_skills_board_section("hot", "Skills Hot", hot)
replacement = f"{trending_sec}\n\n{hot_sec}\n\n"
if _SKILL_SECTIONS.search(md):
return _SKILL_SECTIONS.sub(replacement, md)
github_marker = "🐙 **GitHub Trending"
idx = md.find(github_marker)
if idx >= 0:
return md[:idx] + replacement + md[idx:]
return md.rstrip() + "\n\n" + replacement
def build_wecom_report(
*,
date_str: str,
time_str: str,
updated: str,
highlights: list[str],
theme_line: str,
trending: list[dict[str, Any]],
hot: list[dict[str, Any]],
repos: list[dict[str, Any]],
emerging: list[dict[str, Any]],
topic_name: str,
topic_repos: list[dict[str, Any]],
ai_news: list[dict[str, Any]] | None = None,
pick_command: str,
) -> str:
lines = [
f"{ICONS['header']} **早报 · {date_str}**",
f"> ⏱ {time_str} · 数据截至 {updated}",
"",
f"{ICONS['highlights']} **今日速览**",
]
for point in highlights[:3]:
lines.append(f"> {point}")
lines.append("")
lines.append(f"{ICONS['theme']} {theme_line}")
lines.append("")
if ai_news:
lines.append(f"{ICONS['ainews']} **国际 AI 时讯 Top {len(ai_news)}**")
lines.extend(_ai_news_lines(ai_news))
lines.append("")
lines.append(f"{ICONS['trending']} **Skills Trending Top {len(trending)}**")
for rank, item in enumerate(trending, 1):
lines.extend(_skill_line(rank, item, badge=item.get("badge", "")))
lines.append("")
lines.append(f"{ICONS['hot']} **Skills Hot Top {len(hot)}**")
for rank, item in enumerate(hot, 1):
lines.extend(_skill_line(rank, item, badge=item.get("badge", "")))
lines.append("")
if repos:
lines.append(f"{ICONS['github']} **GitHub Trending Top {len(repos)}**")
lines.extend(_github_repo_lines(repos))
lines.append("")
if emerging:
lines.append(f"{ICONS['emerging']} **新兴项目 Top {len(emerging)}**")
lines.extend(_github_repo_lines(emerging, show_created=True))
lines.append("")
if topic_repos:
lines.append(f"{ICONS['topic']} **Topic `{topic_name}` Top {len(topic_repos)}**")
lines.extend(_github_repo_lines(topic_repos))
lines.append("")
lines.append(f"{ICONS['pick']} **今日首推**")
lines.append(f"`{pick_command}`")
return "\n".join(lines)

635
daily/generate.py Normal file
View File

@@ -0,0 +1,635 @@
"""生成早报 Markdown完整版 + 企微短版)。"""
from __future__ import annotations
import json
import re
import sys
import xml.etree.ElementTree as ET
from collections import defaultdict
from datetime import datetime, timezone, timedelta
from pathlib import Path
from typing import Any
import certifi
import httpx
import logging
logger = logging.getLogger(__name__)
from daily.config import (
CACHE_DIR,
LOG_DIR,
OUTPUT_DIR,
SNAPSHOT_FILE,
ensure_bot_on_path,
env,
env_int,
full_desc_limit,
news_summary_limit,
wecom_skill_desc_limit,
)
from daily.format_wecom import build_wecom_report, finalize_wecom_skill_groups, replace_wecom_skill_sections
from daily.agent_workflow import is_agent_mode, run_agent_workflow
from daily.delta import compare_depth
from daily.cursor_editor import (
apply_descriptions,
is_enabled as cursor_editor_enabled,
run_editorial,
theme_line_from_editorial,
)
from daily.github.auth import github_html_headers
from daily.github.search import fetch_emerging_repos, fetch_topic_hot_repos
from daily.github.trending import fetch_github_trending, trending_data_source_note
from daily.localize import LocalizeJob, localize_descriptions, needs_chinese
from daily.news.fetch import fetch_ai_news, format_news_section, prepare_wecom_news_items
from daily.report_data import (
build_full_payload,
build_llm_input,
data_json_path,
save_json,
)
from daily.skills_board import load_boards
from daily.skills_group import group_skills_by_source
ensure_bot_on_path()
from skills_service import _format_installs, load_feed # noqa: E402
THEME_RULES: list[tuple[str, str, list[str]]] = [
("🎬", "AI 多媒体 / 视频", ["runcomfy", "remotion", "video", "seedance", "inpaint", "lipsync"]),
("🔧", "工程协作 / Skill 元能力", ["grill", "tdd", "architecture", "find-skills", "to-issues"]),
("📱", "飞书 / Lark", ["lark", "feishu"]),
("📣", "内容营销", ["viral", "tiktok", "instagram", "reels"]),
("🎨", "设计 / 前端", ["frontend", "design", "ui-ux", "tailwind"]),
]
def _now_cst() -> datetime:
return datetime.now(timezone(timedelta(hours=8)))
def _short_desc(text: str, limit: int = 72) -> str:
text = re.sub(r"\s+", " ", text or "").strip()
if limit <= 0 or len(text) <= limit:
return text
return text[: limit - 3] + "..."
def _archive_desc(text: str) -> str:
return _short_desc(text, full_desc_limit())
def _wecom_desc(text: str, limit: int = 36) -> str:
return _short_desc(text, limit)
def _localize_descriptions_in_place(
trending: list[dict[str, Any]],
hot: list[dict[str, Any]],
github_trending: list[dict[str, Any]],
github_emerging: list[dict[str, Any]],
github_topic: list[dict[str, Any]],
ai_news: dict[str, Any],
) -> None:
full_limit = full_desc_limit()
news_limit = news_summary_limit()
jobs: list[LocalizeJob] = []
seen_skill: set[str] = set()
for item in trending + hot:
sid = _skill_id(item)
if sid in seen_skill:
continue
seen_skill.add(sid)
desc = (item.get("description") or "").strip()
if desc:
jobs.append(LocalizeJob(f"skill:{sid}", desc, full_limit))
seen_repo: set[str] = set()
for repo_list in (github_trending, github_emerging, github_topic):
for item in repo_list:
repo = item.get("repo", "")
if not repo or repo in seen_repo:
continue
seen_repo.add(repo)
desc = (item.get("description") or "").strip()
if desc:
jobs.append(LocalizeJob(f"github:{repo}", desc, full_limit))
if ai_news.get("enabled"):
seen_news: set[str] = set()
for item in ai_news.get("flat") or []:
link = item.get("link", "")
if not link or link in seen_news:
continue
seen_news.add(link)
summary = (item.get("summary") or "").strip()
if summary:
jobs.append(LocalizeJob(f"news:{link}", summary, news_limit))
zh_map = localize_descriptions(jobs, archive=True)
if not zh_map and not jobs:
return
def _apply_zh(mapping: dict[str, str]) -> None:
for item in trending + hot:
key = f"skill:{_skill_id(item)}"
if key in mapping:
item["description"] = mapping[key]
for repo_list in (github_trending, github_emerging, github_topic):
for item in repo_list:
key = f"github:{item.get('repo', '')}"
if key in mapping:
item["description"] = mapping[key]
if ai_news.get("enabled"):
for cat in ai_news.get("categories") or []:
for item in cat.get("items") or []:
key = f"news:{item.get('link', '')}"
if key in mapping:
item["summary"] = mapping[key]
for item in ai_news.get("flat") or []:
key = f"news:{item.get('link', '')}"
if key in mapping:
item["summary"] = mapping[key]
_apply_zh(zh_map)
# 仍为英文的条目再译一轮(长描述或批次失败时)
retry_jobs: list[LocalizeJob] = []
seen_skill.clear()
for item in trending + hot:
sid = _skill_id(item)
if sid in seen_skill:
continue
seen_skill.add(sid)
desc = (item.get("description") or "").strip()
if needs_chinese(desc):
retry_jobs.append(LocalizeJob(f"skill:{sid}", desc, full_limit))
seen_repo.clear()
for repo_list in (github_trending, github_emerging, github_topic):
for item in repo_list:
repo = item.get("repo", "")
if not repo or repo in seen_repo:
continue
seen_repo.add(repo)
desc = (item.get("description") or "").strip()
if needs_chinese(desc):
retry_jobs.append(LocalizeJob(f"github:{repo}", desc, full_limit))
if ai_news.get("enabled"):
seen_news.clear()
for item in ai_news.get("flat") or []:
link = item.get("link", "")
if not link or link in seen_news:
continue
seen_news.add(link)
summary = (item.get("summary") or "").strip()
if needs_chinese(summary):
retry_jobs.append(LocalizeJob(f"news:{link}", summary, news_limit))
if retry_jobs:
_apply_zh(localize_descriptions(retry_jobs, archive=True))
def _skill_id(item: dict[str, Any]) -> str:
return str(item.get("id") or f"{item.get('source')}/{item.get('title')}")
def _load_snapshot() -> set[str]:
if not SNAPSHOT_FILE.exists():
return set()
try:
data = json.loads(SNAPSHOT_FILE.read_text(encoding="utf-8"))
return set(str(x) for x in (data.get("skill_ids") or []))
except (OSError, json.JSONDecodeError):
return set()
def _save_snapshot(feed: dict[str, Any], date_str: str) -> None:
CACHE_DIR.mkdir(parents=True, exist_ok=True)
ids: list[str] = []
for board in ("topTrending", "topHot"):
for item in feed.get(board, [])[:20]:
sid = _skill_id(item)
if sid not in ids:
ids.append(sid)
SNAPSHOT_FILE.write_text(
json.dumps({"date": date_str, "skill_ids": ids}, ensure_ascii=False, indent=2),
encoding="utf-8",
)
def _prepare_skill_item(item: dict[str, Any], prev_ids: set[str], rank: int) -> dict[str, Any]:
badge = ""
sid = _skill_id(item)
if sid not in prev_ids and prev_ids:
badge = "🆕"
elif rank == 1:
badge = "👑"
installs_fmt = item.get("installs_fmt") or _format_installs(item.get("installs", 0))
title = item.get("source", "?") if item.get("cluster") else item.get("title", "?")
desc = item.get("wecom_desc") or item.get("description") or item.get("cluster_titles") or ""
limit = wecom_skill_desc_limit()
desc_short = desc if item.get("wecom_desc") or limit <= 0 else _wecom_desc(desc, limit)
return {
"title": title,
"source": item.get("source", "?"),
"installs_fmt": installs_fmt,
"link": item.get("link", ""),
"desc_short": desc_short,
"badge": badge,
"cluster": bool(item.get("cluster")),
"cluster_count": item.get("cluster_count"),
"cluster_titles": item.get("cluster_titles"),
}
def _detect_theme_line(feed: dict[str, Any]) -> str:
scores: dict[str, int] = defaultdict(int)
for board in ("topTrending", "topHot"):
for rank, item in enumerate(feed.get(board, [])[:10], 1):
haystack = " ".join(
[item.get("title", ""), item.get("source", ""), item.get("description", "")]
).lower()
for _icon, label, keywords in THEME_RULES:
if any(k in haystack for k in keywords):
scores[label] += max(1, 11 - rank)
break
if not scores:
return "**今日主题**Agent Skills 生态持续活跃"
return f"**今日主题**{max(scores.items(), key=lambda x: x[1])[0]}"
def _build_highlights(
trending: list[dict[str, Any]],
hot: list[dict[str, Any]],
github_trending: list[dict[str, Any]],
github_emerging: list[dict[str, Any]],
ai_news: dict[str, Any] | None = None,
) -> list[str]:
points: list[str] = []
if ai_news and ai_news.get("enabled"):
top_news = prepare_wecom_news_items(ai_news)
if top_news:
n0 = top_news[0]
pub = f" · {n0['published_fmt']}" if n0.get("published_fmt") else ""
points.append(
f"🌍 AI 时讯 [{n0['title']}]({n0['link']})`{n0.get('source_name', '?')}`{pub}"
)
elif ai_news.get("flat"):
n0 = ai_news["flat"][0]
pub = f" · {n0['published_fmt']}" if n0.get("published_fmt") else ""
points.append(f"🌍 AI 时讯 [{n0['title']}]({n0['link']})`{n0.get('source_name', '?')}`{pub}")
if trending:
t0 = trending[0]
points.append(f"📈 Skills 榜首 **{t0.get('title')}**{_format_installs(t0.get('installs', 0))}")
if github_trending:
g0 = github_trending[0]
stars = g0.get("stars_today_fmt", "")
total = g0.get("total_stars_fmt", "")
star_hint = f"+{stars} today · " if stars else (f"{total} · " if total else "")
points.append(f"🐙 GitHub Trending [{g0['repo']}]({g0['url']}){star_hint}{g0.get('language', '')}")
if github_emerging:
e0 = github_emerging[0]
points.append(f"🌱 新兴 [{e0['repo']}]({e0['url']})(⭐ {e0.get('total_stars_fmt', '?')}")
elif hot:
h0 = hot[0]
points.append(f"🔥 Skills Hot 榜首 **{h0.get('title')}**1H {_format_installs(h0.get('installs', 0))}")
while len(points) < 3 and len(trending) > len(points):
item = trending[len(points)]
points.append(f"✨ **{item.get('title')}** · `{item.get('source')}`")
return points[:3]
def _prepare_github_item(item: dict[str, Any]) -> dict[str, Any]:
return {**item, "desc_short": _wecom_desc(item.get("description", ""), 40)}
def _fetch_latest_release_title(repo: str) -> str | None:
atom_url = f"https://github.com/{repo}/releases.atom"
try:
with httpx.Client(
timeout=12.0,
verify=certifi.where(),
follow_redirects=True,
headers=github_html_headers(),
) as client:
resp = client.get(atom_url)
if resp.status_code != 200:
return None
root = ET.fromstring(resp.text)
ns = {"a": "http://www.w3.org/2005/Atom"}
entry = root.find("a:entry", ns)
if entry is None:
return None
title = entry.find("a:title", ns)
return title.text.strip() if title is not None and title.text else None
except Exception:
return None
def _theme_clusters(feed: dict[str, Any], limit: int = 5) -> list[tuple[str, list[str]]]:
buckets: dict[str, list[str]] = defaultdict(list)
seen: set[str] = set()
for board in ("topTrending", "topHot"):
for item in feed.get(board, [])[:20]:
item_id = _skill_id(item)
if item_id in seen:
continue
seen.add(item_id)
haystack = " ".join(
[item.get("title", ""), item.get("source", ""), item.get("description", "")]
).lower()
for _icon, theme, keywords in THEME_RULES:
if any(k in haystack for k in keywords):
label = f"**{item.get('title')}** (`{item.get('source')}`)"
if label not in buckets[theme]:
buckets[theme].append(label)
break
return [(theme, examples[:limit]) for theme, examples in buckets.items() if examples]
def _format_github_repo_section(repos: list[dict[str, Any]], *, show_created: bool = False) -> list[str]:
lines: list[str] = []
for i, repo in enumerate(repos, 1):
lang = repo.get("language") or ""
stars_today = repo.get("stars_today_fmt") or ""
total = repo.get("total_stars_fmt") or ""
created = repo.get("created_at") or ""
meta_parts = [lang]
if stars_today:
meta_parts.append(f"+{stars_today} today")
if total:
meta_parts.append(f"总 ⭐ {total}")
if show_created and created:
meta_parts.append(f"创建于 {created}")
lines.append(f"{i}. **[{repo['repo']}]({repo['url']})** · {' · '.join(meta_parts)}")
desc = _archive_desc(repo.get("description", ""))
if desc:
lines.append(f" - {desc}")
lines.append("")
return lines
def _format_skill_section(items: list[dict[str, Any]], *, hot: bool = False) -> list[str]:
lines: list[str] = []
for i, item in enumerate(items, 1):
skill_id = item.get("id") or f"{item.get('source', '?')}/{item.get('title', '?')}"
link = item.get("link", "")
installs = _format_installs(item.get("installs", 0))
meta = f"1H {installs}" if hot else f"总安装 {installs}"
if link:
lines.append(f"{i}. **[{skill_id}]({link})** · {meta}")
else:
lines.append(f"{i}. **{skill_id}** · {meta}")
desc = _archive_desc(item.get("description", ""))
if desc:
lines.append(f" - {desc}")
lines.append("")
return lines
def generate_report() -> tuple[str, str, Path, Path]:
trending_n = env_int("DAILY_TRENDING_LIMIT", 150)
hot_n = max(env_int("DAILY_HOT_LIMIT", 150), compare_depth())
compare_n = compare_depth()
skill_pool = max(10, env_int("DAILY_WECOM_SKILL_POOL", 200))
wecom_trending = env_int("DAILY_WECOM_TRENDING", 10)
wecom_hot = env_int("DAILY_WECOM_HOT", 10)
github_limit = env_int("DAILY_GITHUB_TRENDING_LIMIT", 10)
wecom_github = env_int("DAILY_WECOM_GITHUB_TRENDING", env_int("DAILY_WECOM_REPOS", 10))
github_fetch_n = max(github_limit, compare_n, wecom_github)
emerging_limit = env_int("DAILY_GITHUB_EMERGING_LIMIT", 10)
wecom_emerging = env_int("DAILY_WECOM_GITHUB_EMERGING", 10)
emerging_fetch_n = max(emerging_limit, compare_n, wecom_emerging)
topic_limit = env_int("DAILY_GITHUB_TOPIC_LIMIT", 10)
wecom_topic = env_int("DAILY_WECOM_GITHUB_TOPIC", 10)
topic_fetch_n = max(topic_limit, compare_n, wecom_topic)
feed = load_feed(force=True)
prev_ids = _load_snapshot()
now = _now_cst()
date_str = now.strftime("%Y-%m-%d")
time_str = now.strftime("%H:%M") + " (UTC+8)"
updated = (feed.get("updatedAt") or "")[:10]
trending, hot = load_boards(feed, trending_limit=trending_n, hot_limit=hot_n)
github_trending = fetch_github_trending(github_fetch_n)
seen_repos = {r["repo"] for r in github_trending}
github_emerging = fetch_emerging_repos(emerging_fetch_n, exclude=seen_repos)
seen_repos.update(r["repo"] for r in github_emerging)
topic_name, github_topic = fetch_topic_hot_repos(topic_fetch_n, exclude=seen_repos)
ai_news = fetch_ai_news()
wecom_limits = {
"trending": wecom_trending,
"hot": wecom_hot,
"trending_pool": skill_pool,
"hot_pool": skill_pool,
"github": wecom_github,
"emerging": wecom_emerging,
"topic": wecom_topic,
"ai_news": env_int("DAILY_WECOM_AI_NEWS", 10),
}
llm_input = build_llm_input(
date_str=date_str,
updated=updated,
trending=trending,
hot=hot,
github_trending=github_trending,
github_emerging=github_emerging,
github_topic=github_topic,
topic_name=topic_name,
ai_news=ai_news,
wecom_limits=wecom_limits,
)
save_json(
data_json_path(date_str),
build_full_payload(
llm_input,
meta={
"generated_at": now.isoformat(),
"report_mode": "agent" if is_agent_mode() else "classic",
"cursor_editor": cursor_editor_enabled() and not is_agent_mode(),
},
),
)
agent_wecom: str | None = None
if is_agent_mode():
agent_wecom = run_agent_workflow(
llm_input,
date_str=date_str,
time_str=time_str,
updated=updated,
)
if not agent_wecom:
logger.warning("Agent 工作流失败,回退 classic 模式")
editorial_theme: str | None = None
editorial_highlights: list[str] | None = None
if agent_wecom is None:
editorial = run_editorial(llm_input, date_str=date_str)
if editorial:
apply_descriptions(
trending=trending,
hot=hot,
github_trending=github_trending,
github_emerging=github_emerging,
github_topic=github_topic,
ai_news=ai_news,
descriptions=editorial.get("descriptions") or {},
)
editorial_theme = theme_line_from_editorial(editorial) or None
hl = editorial.get("highlights") or []
editorial_highlights = hl if hl else None
# 完整版归档:中文化 + 加长摘要(已是中文的条目会跳过翻译)
_localize_descriptions_in_place(
trending, hot, github_trending, github_emerging, github_topic, ai_news
)
themes = _theme_clusters(feed)
lines = [
f"# 早报 · {date_str}",
"",
f"> 生成时间:{now.strftime('%Y-%m-%d %H:%M')} (UTC+8) ",
f"> skills 数据更新:{updated} ",
"> 数据来源:[skills.sh/trending](https://skills.sh/trending) · [skills.sh/hot](https://skills.sh/hot) · 国际 AI RSS",
"",
"---",
"",
f"## 一、Skills Trending Top {trending_n}",
"",
*_format_skill_section(trending),
"---",
"",
f"## 二、Skills Hot Top {hot_n}",
"",
*_format_skill_section(hot, hot=True),
"",
"---",
"",
f"## 三、GitHub Trending Top {github_limit}",
"",
trending_data_source_note(),
"",
]
if github_trending:
lines.extend(_format_github_repo_section(github_trending))
else:
lines.append("*GitHub Trending 获取失败,请检查网络或配置 GITHUB_TOKEN。*")
lines.append("")
lines.extend(["---", "", f"## 四、新兴项目 Top {emerging_limit}", "", "> 数据来源GitHub Search API需 `GITHUB_TOKEN`", ""])
if github_emerging:
lines.extend(_format_github_repo_section(github_emerging, show_created=True))
else:
lines.append("*新兴项目获取失败或未配置 GITHUB_TOKEN。*")
lines.append("")
lines.extend(["---", "", f"## 五、Topic `{topic_name}` Top {topic_limit}", "", "> 数据来源GitHub Search API需 `GITHUB_TOKEN`", ""])
if github_topic:
lines.extend(_format_github_repo_section(github_topic))
else:
lines.append(f"*Topic `{topic_name}` 热点获取失败或未配置 GITHUB_TOKEN。*")
lines.append("")
section_no = 6
lines.extend(format_news_section(ai_news, section_no=section_no))
section_no += 1
watch = (env("GITHUB_REPOS") or "").strip()
if watch:
lines.extend(["---", "", f"## {section_no}、关注仓库 Release", ""])
section_no += 1
for repo in [r.strip() for r in watch.split(",") if r.strip()]:
release = _fetch_latest_release_title(repo)
lines.append(f"- **{repo}**{release or '暂无 release'}")
lines.append("")
lines.extend(["---", "", f"## {section_no}、主题聚类", ""])
for theme, examples in themes:
lines.append(f"### {theme}")
for ex in examples:
lines.append(f"- {ex}")
lines.append("")
pick_src = trending[0].get("source", "") if trending else ""
pick_name = trending[0].get("title", "") if trending else ""
pick_command = (
f"npx skills add {pick_src}/{pick_name}"
if pick_src and pick_name
else "npx skills add vercel-labs/skills/find-skills"
)
lines.extend(["---", "", "## 安装示例", "", "```bash"])
for item in trending[:4]:
src, name = item.get("source", ""), item.get("title", "")
if src and name:
lines.append(f"npx skills add {src}/{name}")
lines.extend(["```", "", f"*企微短版见 `output/{date_str}.wecom.md`*"])
markdown = "\n".join(lines)
if agent_wecom:
gt = group_skills_by_source(trending, limit=wecom_trending, pool_size=skill_pool)
gh = group_skills_by_source(hot, limit=wecom_hot, pool_size=skill_pool)
wecom_md = replace_wecom_skill_sections(agent_wecom, trending=gt, hot=gh)
else:
wecom_md = build_wecom_report(
date_str=date_str,
time_str=time_str,
updated=updated,
highlights=editorial_highlights
or _build_highlights(trending, hot, github_trending, github_emerging, ai_news),
theme_line=editorial_theme or _detect_theme_line(feed),
ai_news=prepare_wecom_news_items(ai_news),
trending=[
_prepare_skill_item(item, prev_ids, r)
for r, item in enumerate(
finalize_wecom_skill_groups(
group_skills_by_source(trending, limit=wecom_trending, pool_size=skill_pool)
),
1,
)
],
hot=[
_prepare_skill_item(item, prev_ids, r)
for r, item in enumerate(
finalize_wecom_skill_groups(
group_skills_by_source(hot, limit=wecom_hot, pool_size=skill_pool)
),
1,
)
],
repos=[_prepare_github_item(item) for item in github_trending[:wecom_github]],
emerging=[_prepare_github_item(item) for item in github_emerging[:wecom_emerging]],
topic_name=topic_name,
topic_repos=[_prepare_github_item(item) for item in github_topic[:wecom_topic]],
pick_command=pick_command,
)
_save_snapshot(feed, date_str)
OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
out_md = OUTPUT_DIR / f"{date_str}.md"
out_wecom = OUTPUT_DIR / f"{date_str}.wecom.md"
out_md.write_text(markdown, encoding="utf-8")
out_wecom.write_text(wecom_md, encoding="utf-8")
return markdown, wecom_md, out_md, out_wecom
def main() -> int:
LOG_DIR.mkdir(parents=True, exist_ok=True)
log_file = LOG_DIR / f"{_now_cst():%Y-%m-%d}.log"
try:
_, wecom_md, out_md, out_wecom = generate_report()
nbytes = len(wecom_md.encode("utf-8"))
msg = f"[{_now_cst():%H:%M:%S}] OK -> {out_md}, {out_wecom} ({nbytes} bytes)\n"
log_file.write_text(msg, encoding="utf-8")
print(msg.strip())
return 0
except Exception as exc:
msg = f"[{_now_cst():%H:%M:%S}] FAIL: {exc}\n"
log_file.write_text(msg, encoding="utf-8")
print(msg.strip(), file=sys.stderr)
return 1

9
daily/github/__init__.py Normal file
View File

@@ -0,0 +1,9 @@
from daily.github.search import fetch_emerging_repos, fetch_topic_hot_repos
from daily.github.trending import fetch_github_trending, trending_data_source_note
__all__ = [
"fetch_emerging_repos",
"fetch_topic_hot_repos",
"fetch_github_trending",
"trending_data_source_note",
]

81
daily/github/auth.py Normal file
View File

@@ -0,0 +1,81 @@
"""GitHub 请求共用GITHUB_TOKEN、请求头、仓库 API。"""
from __future__ import annotations
import logging
from typing import Any
import certifi
import httpx
from daily.config import env
logger = logging.getLogger(__name__)
DEFAULT_USER_AGENT = (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
)
def github_token() -> str | None:
raw = (env("GITHUB_TOKEN") or "").strip()
return raw or None
def github_api_headers() -> dict[str, str]:
headers = {
"User-Agent": env("GITHUB_TRENDING_USER_AGENT", DEFAULT_USER_AGENT),
"Accept": "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28",
}
token = github_token()
if token:
headers["Authorization"] = f"Bearer {token}"
return headers
def github_html_headers() -> dict[str, str]:
headers = {
"User-Agent": env("GITHUB_TRENDING_USER_AGENT", DEFAULT_USER_AGENT),
"Accept": "text/html,application/xhtml+xml",
"Accept-Language": "en-US,en;q=0.9",
}
token = github_token()
if token:
headers["Authorization"] = f"Bearer {token}"
return headers
def fetch_repo_api(repo: str) -> dict[str, Any] | None:
url = f"https://api.github.com/repos/{repo}"
try:
with httpx.Client(
timeout=12.0,
verify=certifi.where(),
headers=github_api_headers(),
) as client:
resp = client.get(url)
if resp.status_code != 200:
return None
data = resp.json()
return {
"description": data.get("description") or "",
"language": data.get("language") or "",
"stars": data.get("stargazers_count", 0),
}
except Exception as exc:
logger.debug("GitHub API repo %s 失败: %s", repo, exc)
return None
def format_star_count(value: int | float | str) -> str:
try:
n = int(value)
except (TypeError, ValueError):
return str(value)
if n >= 1_000_000:
return f"{n / 1_000_000:.1f}M".replace(".0M", "M")
if n >= 1_000:
return f"{n / 1_000:.1f}K".replace(".0K", "K")
return f"{n:,}"

121
daily/github/search.py Normal file
View File

@@ -0,0 +1,121 @@
"""GitHub Search API新兴项目、Topic 热点。"""
from __future__ import annotations
import logging
from datetime import datetime, timedelta, timezone
from typing import Any
import certifi
import httpx
from daily.config import env, env_int
from daily.github.auth import format_star_count, github_api_headers, github_token
logger = logging.getLogger(__name__)
def _repo_from_api_item(item: dict[str, Any], *, source: str) -> dict[str, Any]:
full_name = item.get("full_name") or ""
stars = item.get("stargazers_count", 0)
created = (item.get("created_at") or "")[:10]
return {
"repo": full_name,
"url": item.get("html_url") or f"https://github.com/{full_name}",
"description": item.get("description") or "",
"language": item.get("language") or "",
"stars_today": None,
"stars_today_fmt": "",
"total_stars_fmt": format_star_count(stars),
"created_at": created,
"source": source,
}
def search_github_repos(
query: str,
limit: int,
*,
sort: str = "stars",
require_token: bool = True,
) -> list[dict[str, Any]]:
if require_token and not github_token():
logger.warning("GitHub Search 需要 GITHUB_TOKEN: %s", query[:80])
return []
try:
with httpx.Client(
timeout=20.0,
verify=certifi.where(),
headers=github_api_headers(),
) as client:
resp = client.get(
"https://api.github.com/search/repositories",
params={
"q": query,
"sort": sort,
"order": "desc",
"per_page": min(max(limit, 1), 30),
},
)
if resp.status_code != 200:
logger.warning("GitHub Search 失败 (%s): %s", resp.status_code, query[:80])
return []
items = resp.json().get("items") or []
except Exception as exc:
logger.warning("GitHub Search 异常: %s", exc)
return []
repos: list[dict[str, Any]] = []
for item in items:
full_name = item.get("full_name") or ""
if not full_name:
continue
repos.append(_repo_from_api_item(item, source="api-search"))
if len(repos) >= limit:
break
return repos
def _date_days_ago(days: int) -> str:
return (datetime.now(timezone.utc) - timedelta(days=days)).strftime("%Y-%m-%d")
def fetch_emerging_repos(
limit: int = 3,
*,
days: int | None = None,
min_stars: int | None = None,
exclude: set[str] | None = None,
) -> list[dict[str, Any]]:
days = days if days is not None else env_int("GITHUB_EMERGING_DAYS", 14)
min_stars = min_stars if min_stars is not None else env_int("GITHUB_EMERGING_MIN_STARS", 200)
created_after = _date_days_ago(days)
query = f"created:>{created_after} stars:>{min_stars} fork:false"
repos = search_github_repos(query, limit + len(exclude or set()))
if exclude:
repos = [r for r in repos if r["repo"] not in exclude]
for item in repos:
item["source"] = "api-emerging"
return repos[:limit]
def fetch_topic_hot_repos(
limit: int = 3,
*,
topic: str | None = None,
pushed_days: int | None = None,
min_stars: int | None = None,
exclude: set[str] | None = None,
) -> tuple[str, list[dict[str, Any]]]:
topic = (topic or env("GITHUB_TOPIC") or "llm").strip()
pushed_days = pushed_days if pushed_days is not None else env_int("GITHUB_TOPIC_PUSHED_DAYS", 7)
min_stars = min_stars if min_stars is not None else env_int("GITHUB_TOPIC_MIN_STARS", 50)
pushed_after = _date_days_ago(pushed_days)
query = f"topic:{topic} pushed:>{pushed_after} stars:>{min_stars} fork:false"
repos = search_github_repos(query, limit + len(exclude or set()))
if exclude:
repos = [r for r in repos if r["repo"] not in exclude]
for item in repos:
item["source"] = "api-topic"
return topic, repos[:limit]

206
daily/github/trending.py Normal file
View File

@@ -0,0 +1,206 @@
"""GitHub Trending页面爬取或 Search API。"""
from __future__ import annotations
import logging
import re
from datetime import datetime, timedelta, timezone
from html import unescape
from typing import Any, Literal
from urllib.parse import urlencode
import certifi
import httpx
from daily.config import env
from daily.github.auth import (
fetch_repo_api,
format_star_count,
github_html_headers,
github_token,
)
from daily.github.search import search_github_repos
logger = logging.getLogger(__name__)
TrendingSince = Literal["daily", "weekly", "monthly"]
TrendingMode = Literal["scrape", "api"]
DEFAULT_LIMIT = 5
DEFAULT_SINCE: TrendingSince = "daily"
DEFAULT_MODE: TrendingMode = "scrape"
TRENDING_URL = "https://github.com/trending"
_ARTICLE_RE = re.compile(r'<article class="Box-row">.*?</article>', re.S)
_REPO_HREF_RE = re.compile(r'h2[^>]*>\s*<a[^>]+href="([^"]+)"')
_DESC_RE = re.compile(r'<p class="col-9[^"]*"[^>]*>([^<]*)</p>')
_STARS_TODAY_RE = re.compile(r"([\d,]+)\s+stars?\s+today", re.I)
_LANG_RE = re.compile(r'itemprop="programmingLanguage"[^>]*>([^<]+)<')
_TOTAL_STARS_RE = re.compile(
r'href="/[^/]+/[^/]+/stargazers"[^>]*>\s*<svg[^>]*octicon-star[^>]*>.*?</svg>\s*([\d.,kKmM]+)',
re.S,
)
def _strip_html(text: str) -> str:
return unescape(re.sub(r"\s+", " ", text or "")).strip()
def trending_mode() -> TrendingMode:
raw = (env("GITHUB_TRENDING_MODE") or DEFAULT_MODE).strip().lower()
if raw in {"api", "token", "search"}:
return "api"
return "scrape"
def trending_data_source_note() -> str:
if trending_mode() == "api":
return "> 数据来源GitHub Search API`GITHUB_TRENDING_MODE=api`,需 `GITHUB_TOKEN`"
return "> 数据来源:[github.com/trending](https://github.com/trending?since=daily)(页面爬取,失败时 API 降级)"
def _parse_article(article_html: str) -> dict[str, Any] | None:
href_match = _REPO_HREF_RE.search(article_html)
if not href_match:
return None
href = href_match.group(1).strip("/")
if href.count("/") != 1:
return None
owner, name = href.split("/", 1)
repo = f"{owner}/{name}"
desc_match = _DESC_RE.search(article_html)
stars_today_match = _STARS_TODAY_RE.search(article_html)
lang_match = _LANG_RE.search(article_html)
total_stars_match = _TOTAL_STARS_RE.search(article_html)
stars_today_raw = stars_today_match.group(1).replace(",", "") if stars_today_match else ""
stars_today = int(stars_today_raw) if stars_today_raw.isdigit() else None
return {
"repo": repo,
"url": f"https://github.com/{repo}",
"description": _strip_html(desc_match.group(1)) if desc_match else "",
"language": _strip_html(lang_match.group(1)) if lang_match else "",
"stars_today": stars_today,
"stars_today_fmt": stars_today_match.group(1) if stars_today_match else "",
"total_stars_fmt": _strip_html(total_stars_match.group(1)) if total_stars_match else "",
"source": "scrape",
}
def _build_trending_url(*, since: TrendingSince = DEFAULT_SINCE, language: str = "") -> str:
if language:
return f"{TRENDING_URL}/{language}?{urlencode({'since': since})}"
return f"{TRENDING_URL}?{urlencode({'since': since})}"
def _since_push_date(since: TrendingSince) -> str:
now = datetime.now(timezone.utc)
if since == "weekly":
delta = timedelta(days=7)
elif since == "monthly":
delta = timedelta(days=30)
else:
delta = timedelta(days=1)
return (now - delta).strftime("%Y-%m-%d")
def _enrich_repo_from_api(item: dict[str, Any]) -> dict[str, Any]:
if not github_token() and env("GITHUB_API_ENRICH", "1") != "1":
return item
meta = fetch_repo_api(item["repo"])
if not meta:
return item
enriched = dict(item)
if not enriched.get("description"):
enriched["description"] = meta["description"]
if not enriched.get("language"):
enriched["language"] = meta["language"]
if not enriched.get("total_stars_fmt") and meta["stars"]:
enriched["total_stars_fmt"] = format_star_count(meta["stars"])
enriched["source"] = enriched.get("source", "scrape") + "+api"
return enriched
def _fetch_trending_html(url: str) -> str | None:
try:
with httpx.Client(
timeout=20.0,
verify=certifi.where(),
follow_redirects=True,
headers=github_html_headers(),
) as client:
resp = client.get(url)
if resp.status_code in {403, 429} and not github_token():
logger.warning("GitHub Trending %s(匿名可能被限),可配置 GITHUB_TOKEN", resp.status_code)
resp.raise_for_status()
return resp.text
except Exception as exc:
logger.warning("GitHub Trending 页面抓取失败: %s", exc)
return None
def _parse_trending_html(html: str, limit: int) -> list[dict[str, Any]]:
repos: list[dict[str, Any]] = []
for article_html in _ARTICLE_RE.findall(html):
item = _parse_article(article_html)
if item:
repos.append(_enrich_repo_from_api(item))
if len(repos) >= limit:
break
return repos
def _fetch_trending_via_api(limit: int, since: TrendingSince, language: str) -> list[dict[str, Any]]:
pushed_after = _since_push_date(since)
parts = [f"pushed:>{pushed_after}", "stars:>50", "fork:false"]
if language:
parts.append(f"language:{language}")
query = " ".join(parts)
repos = search_github_repos(query, limit, require_token=True)
for item in repos:
item["source"] = "api-search"
return repos
def fetch_github_trending(
limit: int = DEFAULT_LIMIT,
*,
since: TrendingSince | None = None,
language: str = "",
) -> list[dict[str, Any]]:
since = since or env("GITHUB_TRENDING_SINCE", DEFAULT_SINCE) # type: ignore[assignment]
if since not in ("daily", "weekly", "monthly"):
since = DEFAULT_SINCE
lang = (language or env("GITHUB_TRENDING_LANGUAGE") or "").strip()
mode = trending_mode()
if mode == "api":
repos = _fetch_trending_via_api(limit, since, lang)
if not repos and not github_token():
logger.warning("GITHUB_TRENDING_MODE=api 需要配置 GITHUB_TOKEN")
elif repos:
logger.info("GitHub Trending 使用 API 模式,共 %d", len(repos))
return repos[:limit]
url = _build_trending_url(since=since, language=lang)
html = _fetch_trending_html(url)
repos: list[dict[str, Any]] = []
if html:
repos = _parse_trending_html(html, limit)
if not repos:
logger.warning("GitHub Trending 页面解析为空: %s", url)
if len(repos) < limit:
before = len(repos)
fallback = _fetch_trending_via_api(limit, since, lang)
seen = {r["repo"] for r in repos}
for item in fallback:
if item["repo"] in seen:
continue
repos.append(item)
seen.add(item["repo"])
if len(repos) >= limit:
break
if len(repos) > before:
logger.info("已用 GitHub API 补充 %d 条 Trending 数据", len(repos) - before)
return repos[:limit]

119
daily/llm_client.py Normal file
View File

@@ -0,0 +1,119 @@
"""LLM 调用共享工具OpenAI 兼容 API / Cursor SDK"""
from __future__ import annotations
import json
import logging
import os
import re
from typing import Any
import certifi
import httpx
from daily.config import ROOT, env, env_int
logger = logging.getLogger(__name__)
_JSON_BLOCK = re.compile(r"```(?:json)?\s*([\s\S]*?)```", re.IGNORECASE)
def extract_json_object(text: str) -> dict[str, Any]:
text = text.strip()
if not text:
return {}
try:
data = json.loads(text)
return data if isinstance(data, dict) else {}
except json.JSONDecodeError:
pass
match = _JSON_BLOCK.search(text)
if match:
try:
data = json.loads(match.group(1).strip())
return data if isinstance(data, dict) else {}
except json.JSONDecodeError:
pass
start, end = text.find("{"), text.rfind("}")
if start >= 0 and end > start:
try:
data = json.loads(text[start : end + 1])
return data if isinstance(data, dict) else {}
except json.JSONDecodeError:
pass
return {}
def _openai_chat(system: str, user: str) -> str:
api_key = (env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY") or "").strip()
if not api_key:
return ""
base = (env("DAILY_LLM_API_BASE") or env("OPENAI_API_BASE") or "https://api.openai.com/v1").rstrip("/")
model = env("DAILY_LLM_MODEL") or env("OPENAI_MODEL") or "gpt-4o-mini"
timeout = env_int("DAILY_LLM_TIMEOUT", 120)
payload = {
"model": model,
"temperature": 0.2,
"messages": [
{"role": "system", "content": system},
{"role": "user", "content": user},
],
}
with httpx.Client(timeout=timeout, verify=certifi.where()) as client:
resp = client.post(
f"{base}/chat/completions",
headers={"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"},
json=payload,
)
resp.raise_for_status()
data = resp.json()
return str(data["choices"][0]["message"]["content"] or "").strip()
def _cursor_chat(system: str, user: str) -> str:
api_key = (env("CURSOR_API_KEY") or "").strip()
if not api_key:
return ""
from cursor_sdk import Agent, AgentOptions, CursorAgentError, LocalAgentOptions
from daily.config import ensure_bot_on_path
ensure_bot_on_path()
try:
from bridge_manager import warm_cursor_bridge
except ImportError:
warm_cursor_bridge = lambda: None # noqa: E731
cwd = env("DAILY_CURSOR_CWD") or str(ROOT)
# bridge_manager 读 bot env_config 的 CURSOR_CWD早报侧须先对齐工作目录
os.environ["CURSOR_CWD"] = cwd
warm_cursor_bridge()
model = env("CURSOR_MODEL") or "composer-2.5"
prompt = f"{system}\n\n{user}"
try:
result = Agent.prompt(
prompt,
AgentOptions(
api_key=api_key,
model=model,
local=LocalAgentOptions(cwd=cwd),
),
)
except CursorAgentError as exc:
raise RuntimeError(f"LLM 调用失败:{exc.message}") from exc
if result.status == "error":
raise RuntimeError(f"LLM 调用失败:{result.result or '未知错误'}")
return (result.result or "").strip()
def llm_chat(system: str, user: str) -> str:
"""优先 OpenAI 兼容 API否则 Cursor SDK。"""
if env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY"):
return _openai_chat(system, user)
if env("CURSOR_API_KEY"):
return _cursor_chat(system, user)
return ""
def has_llm_configured() -> bool:
return bool(env("DAILY_LLM_API_KEY") or env("OPENAI_API_KEY") or env("CURSOR_API_KEY"))

236
daily/localize.py Normal file
View File

@@ -0,0 +1,236 @@
"""将英文描述批量改写为简短中文(大模型 + 本地缓存)。"""
from __future__ import annotations
import hashlib
import json
import logging
from dataclasses import dataclass
from daily.config import CACHE_DIR, env, env_int
from daily.cursor_editor import is_enabled as cursor_editor_enabled
from daily.llm_client import extract_json_object, llm_chat
from daily.text_utils import clip_text, trim_brief
logger = logging.getLogger(__name__)
_CACHE_FILE = CACHE_DIR / "zh-desc-cache.json"
@dataclass(frozen=True)
class LocalizeJob:
key: str
text: str
limit: int
def _enabled(*, archive: bool = False) -> bool:
if not archive and cursor_editor_enabled():
return False
raw = (env("DAILY_ZH_DESC") or "1").strip().lower()
return raw not in {"0", "false", "no", "off"}
def _is_mostly_chinese(text: str) -> bool:
text = text.strip()
if not text:
return True
cjk = sum(1 for c in text if "\u4e00" <= c <= "\u9fff")
latin = sum(1 for c in text if c.isascii() and c.isalpha())
return cjk >= max(latin, 1)
def needs_chinese(text: str) -> bool:
"""文本非空且尚未以中文为主。"""
text = (text or "").strip()
if not text:
return False
return not _is_mostly_chinese(text)
def _cache_key(text: str) -> str:
return hashlib.sha1(text.encode("utf-8")).hexdigest()[:16]
def _legacy_cache_key(text: str, limit: int) -> str:
return hashlib.sha1(f"{limit}:{text}".encode("utf-8")).hexdigest()[:16]
def _cache_is_truncated(text: str) -> bool:
t = (text or "").rstrip()
return t.endswith("") or t.endswith("...")
def _lookup_cached(cache: dict[str, str], text: str, limit: int) -> str | None:
hit = cache.get(_cache_key(text))
if hit and not (limit <= 0 and _cache_is_truncated(hit)):
return hit
for legacy_limit in (72, 120, 200):
legacy = cache.get(_legacy_cache_key(text, legacy_limit))
if legacy and not (limit <= 0 and _cache_is_truncated(legacy)):
cache[_cache_key(text)] = legacy
return legacy
return None
def _load_cache() -> dict[str, str]:
if not _CACHE_FILE.exists():
return {}
try:
data = json.loads(_CACHE_FILE.read_text(encoding="utf-8"))
return {str(k): str(v) for k, v in (data.get("entries") or {}).items()}
except (OSError, json.JSONDecodeError, TypeError):
return {}
def _save_cache(entries: dict[str, str]) -> None:
CACHE_DIR.mkdir(parents=True, exist_ok=True)
_CACHE_FILE.write_text(
json.dumps({"entries": entries}, ensure_ascii=False, indent=2),
encoding="utf-8",
)
def _build_prompt(jobs: list[LocalizeJob], *, brief: bool = False) -> tuple[str, str]:
if brief:
system = (
"你是技术早报编辑。把输入 JSON 中每条描述改写为**一句**中文简要介绍。"
"要求:输出必须是中文;只保留核心能力与典型场景;不要逐字翻译;不要加引号或编号;"
"每条必须语义完整、可独立阅读limit 为建议最大字数,请控制在 limit 以内且不要用省略号截断;"
"已是中文且足够简短时可适度精简;"
"只输出 JSON 对象key 与输入一致value 为中文简介字符串。"
)
else:
system = (
"你是技术早报编辑。把输入 JSON 中每条英文描述改写为**完整**中文介绍。"
"要求:输出必须是中文;保留关键能力与使用场景;不要逐字翻译;不要加引号或编号;"
"不要以省略号截断;已是中文则原样或适度精简;"
"只输出 JSON 对象key 与输入一致value 为中文简介字符串。"
)
payload = {job.key: {"text": job.text, "limit": job.limit} for job in jobs}
user = json.dumps(payload, ensure_ascii=False, indent=2)
return system, user
def _brief_cache_key(text: str, limit: int) -> str:
return hashlib.sha1(f"brief:{limit}:{text}".encode("utf-8")).hexdigest()[:16]
def _lookup_brief_cached(cache: dict[str, str], text: str, limit: int) -> str | None:
hit = cache.get(_brief_cache_key(text, limit))
if hit and not _cache_is_truncated(hit):
return hit
return None
def _translate_batch(jobs: list[LocalizeJob], *, brief: bool = False) -> dict[str, str]:
if not jobs:
return {}
system, user = _build_prompt(jobs, brief=brief)
raw = llm_chat(system, user)
if not raw:
return {}
parsed = extract_json_object(raw)
out: dict[str, str] = {}
for job in jobs:
value = parsed.get(job.key)
if isinstance(value, str) and value.strip():
trimmed = value.strip()
if brief and job.limit > 0:
out[job.key] = trim_brief(trimmed, job.limit)
else:
out[job.key] = clip_text(trimmed, job.limit) if job.limit > 0 else trimmed
return out
def localize_brief_descriptions(jobs: list[LocalizeJob], *, archive: bool = False) -> dict[str, str]:
"""企微用:将描述改写为一句简要中文。"""
if not _enabled(archive=archive) or not jobs:
return {}
cache = _load_cache()
result: dict[str, str] = {}
pending: list[LocalizeJob] = []
for job in jobs:
if not job.text.strip():
continue
limit = job.limit if job.limit > 0 else 48
text = job.text.strip()
if _is_mostly_chinese(text) and (limit <= 0 or len(text) <= limit):
result[job.key] = text
continue
cached = _lookup_brief_cached(cache, text, limit)
if cached:
result[job.key] = trim_brief(cached, limit) if limit > 0 else cached
else:
pending.append(LocalizeJob(job.key, text, limit))
if not pending:
return result
batch_size = max(3, min(10, env_int("DAILY_ZH_DESC_BATCH", 20)))
for i in range(0, len(pending), batch_size):
chunk = pending[i : i + batch_size]
try:
translated = _translate_batch(chunk, brief=True)
except Exception as exc:
logger.warning("企微简要摘要批次失败,保留原文:%s", exc)
continue
for job in chunk:
zh = translated.get(job.key)
if not zh:
continue
ck = _brief_cache_key(job.text, job.limit if job.limit > 0 else 48)
cache[ck] = zh
result[job.key] = zh
if cache:
_save_cache(cache)
return result
def localize_descriptions(jobs: list[LocalizeJob], *, archive: bool = False) -> dict[str, str]:
"""返回 job.key -> 中文简介。archive=True 时用于完整版 .md不受 Cursor 编辑层开关影响。"""
if not _enabled(archive=archive) or not jobs:
return {}
cache = _load_cache()
result: dict[str, str] = {}
pending: list[LocalizeJob] = []
for job in jobs:
if not job.text.strip():
continue
if _is_mostly_chinese(job.text):
result[job.key] = clip_text(job.text, job.limit)
continue
ck = _cache_key(job.text)
cached = _lookup_cached(cache, job.text, job.limit)
if cached:
result[job.key] = clip_text(cached, job.limit)
else:
pending.append(job)
if not pending:
return result
batch_size = max(5, env_int("DAILY_ZH_DESC_BATCH", 20))
for i in range(0, len(pending), batch_size):
chunk = pending[i : i + batch_size]
try:
translated = _translate_batch(chunk, brief=False)
except Exception as exc:
logger.warning("中文摘要批次失败,保留英文:%s", exc)
continue
for job in chunk:
zh = translated.get(job.key)
if not zh:
continue
ck = _cache_key(job.text)
cache[ck] = zh
result[job.key] = clip_text(zh, job.limit)
if cache:
_save_cache(cache)
return result

3
daily/news/__init__.py Normal file
View File

@@ -0,0 +1,3 @@
from daily.news.fetch import fetch_ai_news, format_news_section
__all__ = ["fetch_ai_news", "format_news_section"]

124
daily/news/feeds.py Normal file
View File

@@ -0,0 +1,124 @@
"""国际 AI 时讯 RSS 源定义(按类别分组)。"""
from __future__ import annotations
from dataclasses import dataclass
@dataclass(frozen=True)
class NewsFeed:
name: str
url: str
slow: bool = False # 限速源(如 Reddit串行抓取
@dataclass(frozen=True)
class NewsCategory:
id: str
name: str
icon: str
feeds: tuple[NewsFeed, ...]
NEWS_CATEGORIES: tuple[NewsCategory, ...] = (
NewsCategory(
id="official",
name="厂商官方",
icon="🏢",
feeds=(
NewsFeed("Anthropic Claude 更新", "https://docs.anthropic.com/en/release-notes/feed"),
NewsFeed("OpenAI", "https://openai.com/news/rss.xml"),
NewsFeed("Google AI", "https://blog.google/technology/ai/rss/"),
NewsFeed("DeepMind", "https://deepmind.google/blog/rss.xml"),
NewsFeed("Meta Engineering", "https://engineering.fb.com/feed/"),
NewsFeed("Microsoft Research", "https://www.microsoft.com/en-us/research/feed/"),
NewsFeed("Microsoft Blog", "https://blogs.microsoft.com/feed/"),
NewsFeed("Cohere", "https://cohere.com/blog/rss.xml"),
NewsFeed("Cursor Changelog", "https://cursor.com/changelog/rss.xml"),
),
),
NewsCategory(
id="developer",
name="Agent / LLM 开发者",
icon="🛠",
feeds=(
NewsFeed("LangChain", "https://blog.langchain.dev/rss/"),
NewsFeed("Hugging Face", "https://huggingface.co/blog/feed.xml"),
NewsFeed("Vercel Changelog", "https://vercel.com/changelog/rss.xml"),
NewsFeed("GitHub Copilot", "https://github.blog/changelog/label/copilot/feed/"),
),
),
NewsCategory(
id="media",
name="综合科技媒体",
icon="📰",
feeds=(
NewsFeed("The Verge AI", "https://www.theverge.com/rss/ai-artificial-intelligence/index.xml"),
NewsFeed("TechCrunch AI", "https://techcrunch.com/category/artificial-intelligence/feed/"),
NewsFeed("Ars Technica AI", "https://arstechnica.com/ai/feed/"),
NewsFeed("Wired AI", "https://www.wired.com/feed/tag/ai/latest/rss"),
NewsFeed("MIT Tech Review", "https://www.technologyreview.com/feed/"),
NewsFeed("VentureBeat AI", "https://venturebeat.com/category/ai/feed/"),
),
),
NewsCategory(
id="newsletter",
name="Newsletter 日报",
icon="✉️",
feeds=(
NewsFeed("Ben's Bites", "https://bensbites.substack.com/feed"),
NewsFeed("The Rundown AI", "https://therundown.substack.com/feed"),
NewsFeed("Latent Space", "https://www.latent.space/feed"),
NewsFeed("Simon Willison", "https://simonwillison.net/atom/everything/"),
NewsFeed("Import AI", "https://importai.substack.com/feed"),
NewsFeed("Last Week in AI", "https://lastweekin.ai/feed"),
NewsFeed("The Neuron", "https://www.theneuron.ai/feed"),
),
),
NewsCategory(
id="research",
name="研究 / 论文",
icon="📚",
feeds=(
NewsFeed("arXiv cs.CL", "https://arxiv.org/rss/cs.CL"),
NewsFeed("arXiv cs.AI", "https://arxiv.org/rss/cs.AI"),
NewsFeed("arXiv cs.LG", "https://arxiv.org/rss/cs.LG"),
),
),
NewsCategory(
id="trending",
name="热点 / 趋势",
icon="🔥",
feeds=(
NewsFeed(
"Google News · AI",
"https://news.google.com/rss/search?q=artificial+intelligence+OR+LLM+OR+Claude+OR+GPT&hl=en-US&gl=US&ceid=US:en",
),
NewsFeed(
"Google News · Technology",
"https://news.google.com/rss/headlines/section/topic/TECHNOLOGY?hl=en-US&gl=US&ceid=US:en",
),
NewsFeed("Techmeme", "https://www.techmeme.com/feed.xml"),
NewsFeed("HN · Front Page", "https://hnrss.org/frontpage"),
NewsFeed("HN · 100+ Points", "https://hnrss.org/newest?points=100"),
NewsFeed("Dev.to · AI", "https://dev.to/feed/tag/ai"),
NewsFeed("Lobsters", "https://lobste.rs/rss"),
),
),
NewsCategory(
id="community",
name="社区讨论",
icon="💬",
feeds=(
NewsFeed(
"HN · AI/LLM/Agent",
"https://hnrss.org/newest?q=AI+OR+LLM+OR+Claude+OR+agent+OR+GPT+OR+Gemini",
),
NewsFeed(
"Reddit · LLM/Claude/ML",
"https://old.reddit.com/r/LocalLLaMA+ClaudeAI+MachineLearning+OpenAI/.rss?limit=25",
slow=True,
),
),
),
)

462
daily/news/fetch.py Normal file
View File

@@ -0,0 +1,462 @@
"""抓取并整理国际 AI 时讯 RSS。"""
from __future__ import annotations
import logging
import re
import time
import html
import xml.etree.ElementTree as ET
from concurrent.futures import ThreadPoolExecutor, as_completed
from datetime import datetime, timezone, timedelta
from email.utils import parsedate_to_datetime
from typing import Any
from urllib.parse import parse_qs, urlencode, urlparse, urlunparse
import certifi
import httpx
from daily.config import env, env_int, news_summary_limit
from daily.news.feeds import NEWS_CATEGORIES, NewsCategory
logger = logging.getLogger(__name__)
USER_AGENT = "Mozilla/5.0 (compatible; skills-hot-daily/1.0; +https://skills.sh)"
BROWSER_USER_AGENT = (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/131.0.0.0 Safari/537.36"
)
STRIP_HTML = re.compile(r"<[^>]+>")
WS = re.compile(r"\s+")
ATOM_NS = {"a": "http://www.w3.org/2005/Atom"}
RSS_NS = {"r": "http://purl.org/rss/1.0/modules/content/"}
def _enabled() -> bool:
raw = (env("DAILY_AI_NEWS") or "1").strip().lower()
return raw not in {"0", "false", "no", "off"}
def _hours_window() -> int:
return max(1, env_int("DAILY_AI_NEWS_HOURS", 72))
def _per_feed_limit() -> int:
return max(1, env_int("DAILY_AI_NEWS_PER_FEED", 3))
def _per_category_limit() -> int:
return max(1, env_int("DAILY_AI_NEWS_PER_CATEGORY", 5))
def _wecom_limit() -> int:
return max(1, env_int("DAILY_WECOM_AI_NEWS", 10))
def _now_utc() -> datetime:
return datetime.now(timezone.utc)
def _parse_datetime(value: str | None) -> datetime | None:
if not value:
return None
text = value.strip()
if not text:
return None
try:
dt = parsedate_to_datetime(text)
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt.astimezone(timezone.utc)
except (TypeError, ValueError, OverflowError):
pass
for fmt in (
"%Y-%m-%dT%H:%M:%SZ",
"%Y-%m-%dT%H:%M:%S%z",
"%Y-%m-%d",
):
try:
dt = datetime.strptime(text[: len(fmt.replace("%z", "+0000"))], fmt.replace("%z", ""))
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt.astimezone(timezone.utc)
except ValueError:
continue
return None
def _clean_text(text: str | None, limit: int = 200) -> str:
if not text:
return ""
plain = STRIP_HTML.sub(" ", html.unescape(text))
plain = WS.sub(" ", plain).strip()
if limit <= 0 or len(plain) <= limit:
return plain
return plain[: limit - 3] + "..."
def _normalize_link(link: str) -> str:
parsed = urlparse(link.strip())
query = parse_qs(parsed.query, keep_blank_values=False)
for key in list(query.keys()):
if key.lower().startswith("utm_") or key.lower() in {"ref", "source"}:
query.pop(key, None)
clean_query = urlencode({k: v[0] for k, v in query.items() if v}, doseq=False)
return urlunparse((parsed.scheme, parsed.netloc, parsed.path.rstrip("/"), "", clean_query, ""))
def _normalize_title(title: str) -> str:
return WS.sub(" ", title.strip().lower())
def _entry_datetime(entry: dict[str, Any]) -> datetime | None:
for key in ("published", "updated"):
dt = _parse_datetime(entry.get(key))
if dt:
return dt
return None
def _parse_atom(content: str, feed_name: str, category: NewsCategory) -> list[dict[str, Any]]:
items: list[dict[str, Any]] = []
try:
root = ET.fromstring(content)
except ET.ParseError:
return items
for entry in root.findall("a:entry", ATOM_NS):
title_el = entry.find("a:title", ATOM_NS)
link_el = entry.find("a:link", ATOM_NS)
summary_el = entry.find("a:summary", ATOM_NS) or entry.find("a:content", ATOM_NS)
updated_el = entry.find("a:updated", ATOM_NS) or entry.find("a:published", ATOM_NS)
title = title_el.text.strip() if title_el is not None and title_el.text else ""
link = ""
if link_el is not None:
link = link_el.get("href") or (link_el.text or "").strip()
if not title or not link:
continue
items.append(
{
"title": title,
"link": link,
"summary": _clean_text(summary_el.text if summary_el is not None else ""),
"published": updated_el.text.strip() if updated_el is not None and updated_el.text else "",
"source_name": feed_name,
"category_id": category.id,
"category_name": category.name,
"category_icon": category.icon,
}
)
return items
def _parse_rss(content: str, feed_name: str, category: NewsCategory) -> list[dict[str, Any]]:
items: list[dict[str, Any]] = []
try:
root = ET.fromstring(content)
except ET.ParseError:
return items
channel = root.find("channel")
if channel is None:
return items
for item in channel.findall("item"):
title_el = item.find("title")
link_el = item.find("link")
desc_el = item.find("description")
pub_el = item.find("pubDate")
title = title_el.text.strip() if title_el is not None and title_el.text else ""
link = link_el.text.strip() if link_el is not None and link_el.text else ""
if not title or not link:
continue
items.append(
{
"title": title,
"link": link,
"summary": _clean_text(desc_el.text if desc_el is not None else ""),
"published": pub_el.text.strip() if pub_el is not None and pub_el.text else "",
"source_name": feed_name,
"category_id": category.id,
"category_name": category.name,
"category_icon": category.icon,
}
)
return items
def _parse_feed(content: str, feed_name: str, category: NewsCategory) -> list[dict[str, Any]]:
text = content.lstrip("\ufeff").strip()
if not text:
return []
if text.startswith("<rss") or "<channel>" in text[:500]:
return _parse_rss(text, feed_name, category)
if text.startswith("<feed") or "<entry>" in text[:500]:
return _parse_atom(text, feed_name, category)
if "<item>" in text[:2000]:
return _parse_rss(text, feed_name, category)
return _parse_atom(text, feed_name, category)
def _is_reddit_url(url: str) -> bool:
host = urlparse(url).netloc.lower()
return host.endswith("reddit.com")
def _reddit_auth_params() -> dict[str, str]:
user = (env("REDDIT_RSS_USER") or "").strip()
feed = (env("REDDIT_RSS_FEED") or "").strip()
if user and feed:
return {"user": user, "feed": feed}
return {}
def _with_query_params(url: str, extra: dict[str, str]) -> str:
if not extra:
return url
parsed = urlparse(url)
query = parse_qs(parsed.query, keep_blank_values=True)
for key, value in extra.items():
if value and key not in query:
query[key] = [value]
clean_query = urlencode({k: v[0] for k, v in query.items() if v and v[0]}, doseq=False)
return urlunparse((parsed.scheme, parsed.netloc, parsed.path, "", clean_query, ""))
def _reddit_old_url(url: str) -> str:
parsed = urlparse(url)
host = parsed.netloc.lower()
if host.startswith("old."):
return url
if host in {"www.reddit.com", "reddit.com"}:
return urlunparse((parsed.scheme, "old.reddit.com", parsed.path, "", parsed.query, ""))
return url
def _request_headers(base: dict[str, str], url: str) -> dict[str, str]:
if not _is_reddit_url(url):
return base
return {
**base,
"User-Agent": BROWSER_USER_AGENT,
"Accept-Language": "en-US,en;q=0.9",
}
def _reddit_fetch_urls(feed_url: str) -> list[str]:
primary = _with_query_params(feed_url, _reddit_auth_params())
if not _is_reddit_url(primary):
return [primary]
fallback = _reddit_old_url(primary)
if fallback == primary:
return [primary]
return [primary, fallback]
def _fetch_one(client: httpx.Client, category: NewsCategory, feed_url: str, feed_name: str) -> list[dict[str, Any]]:
last_exc: Exception | None = None
for url in _reddit_fetch_urls(feed_url):
try:
headers = _request_headers(dict(client.headers), url)
resp = client.get(url, headers=headers)
resp.raise_for_status()
return _parse_feed(resp.text, feed_name, category)
except Exception as exc:
last_exc = exc
continue
logger.warning("RSS fetch failed [%s] %s: %s", feed_name, feed_url, last_exc)
return []
def _dedupe_items(items: list[dict[str, Any]]) -> list[dict[str, Any]]:
seen_links: set[str] = set()
seen_titles: set[str] = set()
result: list[dict[str, Any]] = []
for item in items:
link_key = _normalize_link(item["link"])
title_key = _normalize_title(item["title"])
if link_key in seen_links or title_key in seen_titles:
continue
seen_links.add(link_key)
seen_titles.add(title_key)
result.append(item)
return result
def _within_window(item: dict[str, Any], cutoff: datetime) -> bool:
dt = _entry_datetime(item)
if dt is None:
return True
return dt >= cutoff
def _sort_key(item: dict[str, Any]) -> tuple[int, datetime]:
dt = _entry_datetime(item)
if dt is None:
return (1, datetime.min.replace(tzinfo=timezone.utc))
return (0, dt)
def fetch_ai_news() -> dict[str, Any]:
"""按类别抓取 AI 时讯,返回 {enabled, hours, categories, flat, stats}。"""
if not _enabled():
return {"enabled": False, "categories": [], "flat": [], "stats": {}}
hours = _hours_window()
per_feed = _per_feed_limit()
per_category = _per_category_limit()
cutoff = _now_utc() - timedelta(hours=hours)
headers = {"User-Agent": USER_AGENT, "Accept": "application/rss+xml, application/atom+xml, application/xml, text/xml, */*"}
tasks: list[tuple[NewsCategory, str, str, bool]] = []
for category in NEWS_CATEGORIES:
for feed in category.feeds:
tasks.append((category, feed.url, feed.name, feed.slow))
raw_by_category: dict[str, list[dict[str, Any]]] = {c.id: [] for c in NEWS_CATEGORIES}
stats = {"feeds_total": len(tasks), "feeds_ok": 0, "items_raw": 0}
with httpx.Client(timeout=15.0, verify=certifi.where(), follow_redirects=True, headers=headers) as client:
fast_tasks = [t for t in tasks if not t[3]]
slow_tasks = [t for t in tasks if t[3]]
with ThreadPoolExecutor(max_workers=8) as pool:
futures = {
pool.submit(_fetch_one, client, cat, url, name): (cat.id, name)
for cat, url, name, _slow in fast_tasks
}
for future in as_completed(futures):
cat_id, feed_name = futures[future]
try:
entries = future.result()
except Exception as exc:
logger.warning("RSS 任务异常 [%s]: %s", feed_name, exc)
continue
if entries:
stats["feeds_ok"] += 1
stats["items_raw"] += len(entries)
raw_by_category[cat_id].extend(entries[:per_feed])
for cat, url, name, _slow in slow_tasks:
entries = _fetch_one(client, cat, url, name)
if entries:
stats["feeds_ok"] += 1
stats["items_raw"] += len(entries)
raw_by_category[cat.id].extend(entries[:per_feed])
if _is_reddit_url(url):
time.sleep(2.0)
else:
time.sleep(1.0)
categories_out: list[dict[str, Any]] = []
flat: list[dict[str, Any]] = []
for category in NEWS_CATEGORIES:
items = raw_by_category[category.id]
items = [i for i in items if _within_window(i, cutoff)]
items.sort(key=_sort_key, reverse=True)
items = _dedupe_items(items)[:per_category]
for item in items:
dt = _entry_datetime(item)
item["published_fmt"] = dt.astimezone(timezone(timedelta(hours=8))).strftime("%m-%d %H:%M") if dt else ""
if items:
categories_out.append(
{
"id": category.id,
"name": category.name,
"icon": category.icon,
"items": items,
}
)
flat.extend(items)
flat.sort(key=_sort_key, reverse=True)
flat = _dedupe_items(flat)
return {
"enabled": True,
"hours": hours,
"categories": categories_out,
"flat": flat,
"stats": stats,
}
def format_news_section(news: dict[str, Any], *, section_no: int, wecom_limit: int | None = None) -> list[str]:
if not news.get("enabled"):
return ["---", "", f"## {section_no}、国际 AI 时讯", "", "*AI 时讯已关闭(`DAILY_AI_NEWS=0`)。*", ""]
categories = news.get("categories") or []
hours = news.get("hours", 72)
lines = [
"---",
"",
f"## {section_no}、国际 AI 时讯",
"",
f"> 近 **{hours}h** · {news.get('stats', {}).get('feeds_ok', 0)}/{news.get('stats', {}).get('feeds_total', 0)} 源可用",
"",
]
if not categories:
lines.append("*暂无可用条目(网络/RSS 源异常或时间窗口内无更新)。*")
lines.append("")
return lines
if wecom_limit is not None:
flat = (news.get("flat") or [])[:wecom_limit]
for i, item in enumerate(flat, 1):
pub = f" · {item['published_fmt']}" if item.get("published_fmt") else ""
lines.append(
f"{i}. [{item['title']}]({item['link']}) · `{item['source_name']}`{pub}"
)
lines.append("")
return lines
for cat in categories:
lines.append(f"### {cat['icon']} {cat['name']}")
lines.append("")
for i, item in enumerate(cat["items"], 1):
pub = f" · {item['published_fmt']}" if item.get("published_fmt") else ""
lines.append(f"{i}. **[{item['title']}]({item['link']})** · `{item['source_name']}`{pub}")
summary = item.get("summary", "")
if summary:
lines.append(f" - {_clean_text(summary, news_summary_limit())}")
lines.append("")
return lines
def prepare_wecom_news_items(news: dict[str, Any]) -> list[dict[str, Any]]:
if not news.get("enabled"):
return []
limit = _wecom_limit()
flat = _dedupe_items(news.get("flat") or [])
flat.sort(key=_sort_key, reverse=True)
preferred = ("media", "newsletter", "official", "community", "research", "developer")
picked: list[dict[str, Any]] = []
seen: set[str] = set()
for cat in preferred:
for item in flat:
link = _normalize_link(item.get("link", ""))
if item.get("category_id") != cat or link in seen:
continue
picked.append(item)
seen.add(link)
if len(picked) >= limit:
break
if len(picked) >= limit:
break
items: list[dict[str, Any]] = []
for item in picked[:limit]:
items.append(
{
"title": item.get("title", "?"),
"link": item.get("link", ""),
"source_name": item.get("source_name", "?"),
"published_fmt": item.get("published_fmt", ""),
"desc_short": _clean_text(item.get("summary", ""), 36),
}
)
return items

183
daily/report_data.py Normal file
View File

@@ -0,0 +1,183 @@
"""早报结构化数据:抓取结果 → JSON 中间层。"""
from __future__ import annotations
import json
from pathlib import Path
from typing import Any
from daily.config import OUTPUT_DIR, env_int
from daily.delta import build_movement_baseline, build_movement_context, compare_depth
from daily.news.fetch import prepare_wecom_news_items
from daily.skills_group import group_skills_by_source
def skill_id(item: dict[str, Any]) -> str:
return str(item.get("id") or f"{item.get('source')}/{item.get('title')}")
def _slim_skill(item: dict[str, Any]) -> dict[str, Any]:
payload = {
"id": skill_id(item),
"title": item.get("title", ""),
"source": item.get("source", ""),
"installs": item.get("installs", 0),
"link": item.get("link", ""),
"description": item.get("description", ""),
}
if item.get("cluster"):
payload.update(
{
"cluster": True,
"cluster_count": item.get("cluster_count", 1),
"cluster_skills": item.get("cluster_skills", []),
"cluster_titles": item.get("cluster_titles", ""),
"installs_fmt": item.get("installs_fmt", ""),
"installs_min": item.get("installs_min"),
"installs_max": item.get("installs_max"),
}
)
elif item.get("installs_fmt"):
payload["installs_fmt"] = item.get("installs_fmt")
return payload
def _slim_github(item: dict[str, Any]) -> dict[str, Any]:
return {
"repo": item.get("repo", ""),
"url": item.get("url", ""),
"language": item.get("language", ""),
"stars_today_fmt": item.get("stars_today_fmt", ""),
"total_stars_fmt": item.get("total_stars_fmt", ""),
"created_at": item.get("created_at", ""),
"description": item.get("description", ""),
}
def _slim_news_items(ai_news: dict[str, Any], limit: int) -> list[dict[str, Any]]:
items: list[dict[str, Any]] = []
for item in prepare_wecom_news_items(ai_news):
items.append(
{
"link": item.get("link", ""),
"title": item.get("title", ""),
"source_name": item.get("source_name", ""),
"published_fmt": item.get("published_fmt", ""),
"summary": item.get("desc_short") or "",
}
)
if len(items) >= limit:
break
if items:
return items
for item in (ai_news.get("flat") or [])[:limit]:
items.append(
{
"link": item.get("link", ""),
"title": item.get("title", ""),
"source_name": item.get("source_name", ""),
"published_fmt": item.get("published_fmt", ""),
"summary": item.get("summary", ""),
}
)
return items
def _wecom_skill_pool() -> int:
return max(10, env_int("DAILY_WECOM_SKILL_POOL", 200))
def build_llm_input(
*,
date_str: str,
updated: str,
trending: list[dict[str, Any]],
hot: list[dict[str, Any]],
github_trending: list[dict[str, Any]],
github_emerging: list[dict[str, Any]],
github_topic: list[dict[str, Any]],
topic_name: str,
ai_news: dict[str, Any],
wecom_limits: dict[str, int],
) -> dict[str, Any]:
"""供 Cursor 编辑的精简 JSON不含完整 markdown"""
news_limit = wecom_limits.get("ai_news", 10)
depth = compare_depth()
trend_cmp = trending[:depth]
hot_cmp = hot[:depth]
github_cmp = github_trending[:depth]
emerging_cmp = github_emerging[:depth]
topic_cmp = github_topic[:depth]
trending_slice = group_skills_by_source(
trending,
limit=wecom_limits.get("trending", 10),
pool_size=wecom_limits.get("trending_pool", _wecom_skill_pool()),
)
hot_slice = group_skills_by_source(
hot,
limit=wecom_limits.get("hot", 10),
pool_size=wecom_limits.get("hot_pool", _wecom_skill_pool()),
)
github_slice = github_trending[: wecom_limits.get("github", 5)]
emerging_slice = github_emerging[: wecom_limits.get("emerging", 3)]
topic_slice = github_topic[: wecom_limits.get("topic", 3)]
movement = build_movement_context(
date_str=date_str,
trending=[_slim_skill(x) for x in trend_cmp],
hot=[_slim_skill(x) for x in hot_cmp],
github_trending=[_slim_github(x) for x in github_cmp],
github_emerging=[_slim_github(x) for x in emerging_cmp],
github_topic=[_slim_github(x) for x in topic_cmp],
topic_name=topic_name,
)
movement_baseline = build_movement_baseline(
trending=[_slim_skill(x) for x in trend_cmp],
hot=[_slim_skill(x) for x in hot_cmp],
github_trending=[_slim_github(x) for x in github_cmp],
github_emerging=[_slim_github(x) for x in emerging_cmp],
github_topic=[_slim_github(x) for x in topic_cmp],
depth=depth,
)
return {
"date": date_str,
"data_updated": updated,
"skills_trending": [_slim_skill(x) for x in trending_slice],
"skills_hot": [_slim_skill(x) for x in hot_slice],
"github_trending": [_slim_github(x) for x in github_slice],
"github_emerging": [_slim_github(x) for x in emerging_slice],
"github_topic": {
"topic": topic_name,
"repos": [_slim_github(x) for x in topic_slice],
},
"ai_news": _slim_news_items(ai_news, news_limit) if ai_news.get("enabled") else [],
"movement": movement,
"movement_baseline": movement_baseline,
}
def build_full_payload(
llm_input: dict[str, Any],
*,
meta: dict[str, Any],
) -> dict[str, Any]:
return {"meta": meta, "data": llm_input}
def data_json_path(date_str: str) -> Path:
return OUTPUT_DIR / f"{date_str}.data.json"
def editorial_json_path(date_str: str) -> Path:
return OUTPUT_DIR / f"{date_str}.editorial.json"
def save_json(path: Path, data: dict[str, Any]) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding="utf-8")
def load_json(path: Path) -> dict[str, Any]:
return json.loads(path.read_text(encoding="utf-8"))

120
daily/skills_board.py Normal file
View File

@@ -0,0 +1,120 @@
"""从 skills.sh 官网抓取 Trending / Hot 完整榜单(突破 feed.json 50 条限制)。"""
from __future__ import annotations
import logging
import re
from typing import Any, Literal
import certifi
import httpx
from daily.config import env
logger = logging.getLogger(__name__)
Board = Literal["trending", "hot"]
SKILLS_SITE = "https://www.skills.sh"
USER_AGENT = "Mozilla/5.0 (compatible; skills-hot-daily/1.0; +https://skills.sh)"
_SKILL_RE = re.compile(
r'\{"source":"(?P<source>[^"]+)","skillId":"(?P<skill_id>[^"]+)",'
r'"name":"(?P<name>[^"]+)","installs":(?P<installs>\d+)'
)
_RSC_CHUNK_RE = re.compile(r"self\.__next_f\.push\(\[1,\"(.*?)\"\]\)", re.DOTALL)
def board_source() -> str:
return (env("SKILLS_BOARD_SOURCE") or "website").strip().lower()
def _fetch_html(path: str) -> str:
url = f"{SKILLS_SITE}{path}"
headers = {"User-Agent": USER_AGENT, "Accept": "text/html"}
with httpx.Client(timeout=30.0, verify=certifi.where(), follow_redirects=True) as client:
resp = client.get(url, headers=headers)
resp.raise_for_status()
return resp.text
def _rsc_blob(html: str) -> str:
chunks = _RSC_CHUNK_RE.findall(html)
blob = "\n".join(chunks)
return blob.encode("utf-8").decode("unicode_escape", errors="ignore")
def _parse_initial_skills(blob: str, *, limit: int) -> list[dict[str, Any]]:
items: list[dict[str, Any]] = []
seen: set[str] = set()
for match in _SKILL_RE.finditer(blob):
source = match.group("source")
skill_id = match.group("skill_id")
uid = f"{source}/{skill_id}"
if uid in seen:
continue
seen.add(uid)
items.append(
{
"id": uid,
"title": skill_id,
"source": source,
"installs": int(match.group("installs")),
"link": f"{SKILLS_SITE}/{source}/{skill_id}",
"description": "",
}
)
if len(items) >= limit:
break
return items
def fetch_board(board: Board, *, limit: int) -> list[dict[str, Any]]:
path = "/trending" if board == "trending" else "/hot"
try:
html = _fetch_html(path)
items = _parse_initial_skills(_rsc_blob(html), limit=limit)
if items:
logger.info("skills.sh %s: %d items (limit=%d)", board, len(items), limit)
return items
except Exception as exc:
logger.warning("skills.sh %s fetch failed, fallback to feed.json: %s", board, exc)
return []
def enrich_from_feed(items: list[dict[str, Any]], feed: dict[str, Any]) -> None:
desc_by_id: dict[str, str] = {}
for key in ("topTrending", "topHot", "topAllTime"):
for row in feed.get(key, []):
uid = str(row.get("id") or f"{row.get('source')}/{row.get('title')}")
desc = (row.get("description") or "").strip()
if desc:
desc_by_id[uid] = desc
for item in items:
if not item.get("description"):
item["description"] = desc_by_id.get(item["id"], "")
def load_boards(
feed: dict[str, Any],
*,
trending_limit: int,
hot_limit: int,
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
"""优先 skills.sh 官网;失败时回退 feed.json。"""
if board_source() == "feed":
return (
list(feed.get("topTrending", [])[:trending_limit]),
list(feed.get("topHot", [])[:hot_limit]),
)
trending = fetch_board("trending", limit=trending_limit)
hot = fetch_board("hot", limit=hot_limit)
if not trending:
trending = list(feed.get("topTrending", [])[:trending_limit])
else:
enrich_from_feed(trending, feed)
if not hot:
hot = list(feed.get("topHot", [])[:hot_limit])
else:
enrich_from_feed(hot, feed)
return trending, hot

103
daily/skills_group.py Normal file
View File

@@ -0,0 +1,103 @@
"""Skills 榜单:同 source 合并为一条(企微 Top N"""
from __future__ import annotations
from typing import Any
def skill_id(item: dict[str, Any]) -> str:
return str(item.get("id") or f"{item.get('source')}/{item.get('title')}")
def format_installs(n: int | float) -> str:
if n >= 1_000_000:
return f"{n / 1_000_000:.1f}M"
if n >= 1_000:
return f"{n / 1_000:.1f}K"
return str(int(n))
def _installs_range(items: list[dict[str, Any]]) -> tuple[int, int, str]:
values = [int(item.get("installs") or 0) for item in items]
lo, hi = min(values), max(values)
if lo == hi:
return lo, hi, format_installs(hi)
return lo, hi, f"{format_installs(lo)}{format_installs(hi)}"
def _best_description(items: list[dict[str, Any]]) -> str:
for item in items:
desc = (item.get("description") or "").strip()
if desc:
return desc
return ""
def _cluster_skill(items: list[dict[str, Any]]) -> dict[str, Any]:
ranked = sorted(items, key=lambda x: int(x.get("installs") or 0), reverse=True)
top = ranked[0]
_lo, _hi, installs_fmt = _installs_range(ranked)
titles = [str(x.get("title") or "") for x in ranked if x.get("title")]
sample = ", ".join(titles[:4])
if len(titles) > 4:
sample = f"{sample}"
return {
"id": skill_id(top),
"title": top.get("title", ""),
"source": top.get("source", ""),
"installs": int(top.get("installs") or 0),
"installs_min": _lo,
"installs_max": _hi,
"installs_fmt": installs_fmt,
"link": top.get("link", ""),
"description": _best_description(ranked),
"cluster": True,
"cluster_count": len(ranked),
"cluster_skills": titles,
"cluster_titles": sample,
}
def _single_skill(item: dict[str, Any]) -> dict[str, Any]:
installs = int(item.get("installs") or 0)
return {
"id": skill_id(item),
"title": item.get("title", ""),
"source": item.get("source", ""),
"installs": installs,
"installs_fmt": format_installs(installs),
"link": item.get("link", ""),
"description": item.get("description", ""),
"cluster": False,
}
def group_skills_by_source(
items: list[dict[str, Any]],
*,
limit: int = 10,
pool_size: int = 50,
) -> list[dict[str, Any]]:
"""按 source 去重合并;保留各 source 在榜内的最佳名次顺序。"""
if not items or limit <= 0:
return []
pool = items[: max(pool_size, limit)]
by_source: dict[str, list[dict[str, Any]]] = {}
first_rank: dict[str, int] = {}
for rank, item in enumerate(pool, 1):
source = (item.get("source") or "?").strip() or "?"
by_source.setdefault(source, []).append(item)
first_rank.setdefault(source, rank)
ordered_sources = sorted(by_source.keys(), key=lambda s: first_rank[s])
result: list[dict[str, Any]] = []
for source in ordered_sources:
group = by_source[source]
if len(group) == 1:
result.append(_single_skill(group[0]))
else:
result.append(_cluster_skill(group))
if len(result) >= limit:
break
return result

30
daily/text_utils.py Normal file
View File

@@ -0,0 +1,30 @@
"""文本裁剪等轻量工具。"""
from __future__ import annotations
import re
_WS = re.compile(r"\s+")
def clip_text(text: str, limit: int) -> str:
text = _WS.sub(" ", (text or "").strip())
if limit <= 0 or len(text) <= limit:
return text
return text[: max(1, limit - 1)] + ""
def trim_brief(text: str, limit: int) -> str:
"""企微简要:控制在 limit 内,优先在句读处截断,不加省略号。"""
text = _WS.sub(" ", (text or "").strip())
if not text or limit <= 0 or len(text) <= limit:
return text
for sep in ("", "", "", ""):
pos = text.find(sep)
if pos != -1 and pos + 1 <= limit + 8:
return text[: pos + 1]
for sep in ("", ""):
pos = text.find(sep)
if pos != -1 and pos + 1 <= limit:
return text[: pos + 1]
return text[:limit].rstrip(",、;: ")

96
daily/webhook.py Normal file
View File

@@ -0,0 +1,96 @@
"""推送早报至企业微信群 webhook超长自动分多条"""
from __future__ import annotations
import sys
import time
from pathlib import Path
import certifi
import httpx
from daily.config import OUTPUT_DIR, ROOT, env_int, wecom_chunk_bytes
from daily.wecom_split import split_wecom_messages
_PUSH_GAP_MS = 300
def _load_webhook_key() -> str:
from daily.config import env
key = (env("WECOM_WEBHOOK_KEY") or "").strip()
if not key:
raise RuntimeError("请设置 WECOM_WEBHOOK_KEY项目根 .env")
return key
def _resolve_report_path(arg: str | None) -> Path:
if arg:
path = Path(arg)
if not path.is_absolute():
path = ROOT / path
return path
candidates = sorted(
OUTPUT_DIR.glob("*.wecom.md"),
key=lambda p: p.stat().st_mtime,
reverse=True,
)
if candidates:
return candidates[0]
legacy = sorted(ROOT.glob("*.wecom.md"), key=lambda p: p.stat().st_mtime, reverse=True)
if legacy:
return legacy[0]
raise RuntimeError("未找到 .wecom.md 报告,请先运行 python -m daily")
def _post_markdown(client: httpx.Client, url: str, content: str) -> None:
payload = {"msgtype": "markdown", "markdown": {"content": content}}
resp = client.post(url, json=payload)
resp.raise_for_status()
data = resp.json()
if data.get("errcode", 0) != 0:
raise RuntimeError(f"推送失败: errcode={data.get('errcode')} errmsg={data.get('errmsg')}")
def send_report(report_path: Path | None = None) -> None:
path = _resolve_report_path(str(report_path) if report_path else None)
if not path.exists():
raise RuntimeError(f"报告文件不存在: {path}")
content = path.read_text(encoding="utf-8").strip()
if not content:
raise RuntimeError(f"报告内容为空: {path}")
chunk_limit = wecom_chunk_bytes()
max_parts = env_int("DAILY_WECOM_MAX_PARTS", 5)
parts = split_wecom_messages(content, chunk_limit)
if len(parts) > max_parts:
raise RuntimeError(
f"早报需 {len(parts)} 条消息,超过 DAILY_WECOM_MAX_PARTS={max_parts},请调小各区块条数"
)
key = _load_webhook_key()
url = f"https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key={key}"
with httpx.Client(timeout=20.0, verify=certifi.where()) as client:
for i, part in enumerate(parts, 1):
if i > 1:
time.sleep(_PUSH_GAP_MS / 1000.0)
_post_markdown(client, url, part)
total_bytes = len(content.encode("utf-8"))
if len(parts) == 1:
print(f"已推送至企业微信: {path.name} ({total_bytes} bytes)")
else:
sizes = ", ".join(str(len(p.encode("utf-8"))) for p in parts)
print(f"已推送至企业微信: {path.name} ({total_bytes} bytes → {len(parts)} 条: {sizes})")
def main(argv: list[str] | None = None) -> int:
args = argv if argv is not None else sys.argv[1:]
try:
send_report(Path(args[0]) if args else None)
return 0
except Exception as exc:
print(f"ERROR: {exc}", file=sys.stderr)
return 1

94
daily/wecom_split.py Normal file
View File

@@ -0,0 +1,94 @@
"""企微 markdown 按字节上限拆分为多条消息(按区块,不截断正文)。"""
from __future__ import annotations
import re
_SECTION_START = re.compile(r"^[📰💡🎯🌍📈🔥🐙🌱🤖📦📄]")
def _utf8_len(text: str) -> int:
return len(text.encode("utf-8"))
def _split_lines_by_budget(text: str, limit: int) -> list[str]:
lines = text.splitlines()
chunks: list[str] = []
buf: list[str] = []
for line in lines:
candidate = "\n".join(buf + [line]) if buf else line
if _utf8_len(candidate) <= limit:
buf.append(line)
continue
if buf:
chunks.append("\n".join(buf))
buf = []
if _utf8_len(line) <= limit:
buf = [line]
else:
encoded = line.encode("utf-8")
start = 0
while start < len(encoded):
piece = encoded[start : start + limit].decode("utf-8", errors="ignore")
chunks.append(piece)
start += len(piece.encode("utf-8"))
if buf:
chunks.append("\n".join(buf))
return chunks
def _split_sections(text: str) -> list[str]:
sections: list[str] = []
current: list[str] = []
for line in text.splitlines():
if _SECTION_START.match(line) and current:
sections.append("\n".join(current))
current = [line]
else:
current.append(line)
if current:
sections.append("\n".join(current))
return sections
def split_wecom_messages(text: str, limit: int = 4096) -> list[str]:
"""超长时拆成多条;每条不超过 limit 字节,按区块边界优先。"""
text = text.strip()
if not text or _utf8_len(text) <= limit:
return [text] if text else []
footer_reserve = 40
pack_limit = max(512, limit - footer_reserve)
sections: list[str] = []
for sec in _split_sections(text):
if _utf8_len(sec) <= pack_limit:
sections.append(sec)
else:
sections.extend(_split_lines_by_budget(sec, pack_limit))
packed: list[str] = []
buf: list[str] = []
for sec in sections:
candidate = "\n\n".join(buf + [sec]) if buf else sec
if _utf8_len(candidate) <= pack_limit:
buf.append(sec)
else:
if buf:
packed.append("\n\n".join(buf))
buf = [sec]
if buf:
packed.append("\n\n".join(buf))
total = len(packed)
if total <= 1:
return packed
result: list[str] = []
for i, chunk in enumerate(packed, 1):
suffix = f"\n\n> 📄 {i}/{total}"
body = chunk
while body and _utf8_len(body + suffix) > limit:
body = body.rsplit("\n", 1)[0] if "\n" in body else body[:-1]
result.append(body + suffix)
return result

5002
pre.json Normal file

File diff suppressed because it is too large Load Diff

72
register-daily-task.ps1 Normal file
View File

@@ -0,0 +1,72 @@
# Register a Windows scheduled task to run run-daily.ps1
# Usage:
# .\register-daily-task.ps1
# .\register-daily-task.ps1 -Time "08:50"
# .\register-daily-task.ps1 -Unregister
param(
[string]$Time = "08:50",
[string]$TaskName = "DailyRobots",
[switch]$Unregister
)
$ErrorActionPreference = "Stop"
$Root = Split-Path -Parent $MyInvocation.MyCommand.Path
$RunScript = Join-Path $Root "run-daily.ps1"
$LogDir = Join-Path $Root "logs"
$LogFile = Join-Path $LogDir "scheduled-run.log"
if ($Unregister) {
Unregister-ScheduledTask -TaskName $TaskName -Confirm:$false -ErrorAction SilentlyContinue
Write-Host "Removed scheduled task: $TaskName"
exit 0
}
if (-not (Test-Path $RunScript)) {
throw "Not found: $RunScript"
}
if (-not (Test-Path $LogDir)) {
New-Item -ItemType Directory -Path $LogDir | Out-Null
}
# Append stdout/stderr to logs/scheduled-run.log for troubleshooting
$Argument = @(
"-NoProfile",
"-ExecutionPolicy", "Bypass",
"-Command",
"& { Set-Location '$Root'; & '$RunScript' *>&1 | Tee-Object -FilePath '$LogFile' -Append; exit `$LASTEXITCODE }"
) -join " "
$Action = New-ScheduledTaskAction `
-Execute "powershell.exe" `
-Argument $Argument `
-WorkingDirectory $Root
$Trigger = New-ScheduledTaskTrigger -Daily -At $Time
$Settings = New-ScheduledTaskSettingsSet `
-AllowStartIfOnBatteries `
-DontStopIfGoingOnBatteries `
-StartWhenAvailable `
-ExecutionTimeLimit (New-TimeSpan -Hours 2)
$Principal = New-ScheduledTaskPrincipal -UserId $env:USERNAME -LogonType Interactive -RunLevel Limited
Register-ScheduledTask `
-TaskName $TaskName `
-Action $Action `
-Trigger $Trigger `
-Settings $Settings `
-Principal $Principal `
-Description "Generate and push daily report (run-daily.ps1)" `
-Force | Out-Null
Write-Host "Scheduled task registered:"
Write-Host " Name: $TaskName"
Write-Host " Time: daily at $Time"
Write-Host " Script: $RunScript"
Write-Host " Log: $LogFile"
Write-Host ""
Write-Host "Test now: Start-ScheduledTask -TaskName '$TaskName'"
Write-Host "Remove: .\register-daily-task.ps1 -Unregister"

5
requirements.txt Normal file
View File

@@ -0,0 +1,5 @@
python-dotenv>=1.0.0
httpx>=0.27.0
certifi>=2024.0.0
# 英文描述转中文(使用 CURSOR_API_KEY 时需安装)
cursor-sdk>=0.1.0

58
run-daily.ps1 Normal file
View File

@@ -0,0 +1,58 @@
# Daily report: generate + push to WeCom webhook
# Usage:
# .\run-daily.ps1
# .\run-daily.ps1 -SkipPush
# .\run-daily.ps1 -SkipGenerate
param(
[switch]$SkipPush,
[switch]$SkipGenerate
)
$ErrorActionPreference = "Stop"
$Root = Split-Path -Parent $MyInvocation.MyCommand.Path
Set-Location $Root
function Import-DotEnvFile {
param([string]$Path)
if (-not (Test-Path $Path)) { return }
Get-Content $Path -Encoding UTF8 | ForEach-Object {
if ($_ -match '^\s*#' -or $_ -notmatch '=') { return }
$pair = $_ -split '=', 2
if ($pair.Count -eq 2) {
$name = $pair[0].Trim()
$value = $pair[1].Trim().Trim('"').Trim("'")
if ($name -and $value) {
Set-Item -Path "Env:$name" -Value $value
}
}
}
}
Import-DotEnvFile (Join-Path $Root ".env")
Import-DotEnvFile (Join-Path $Root ".env.local")
$python = "python"
$date = Get-Date -Format "yyyy-MM-dd"
$reportWecom = Join-Path $Root "output\$date.wecom.md"
if (-not $SkipGenerate) {
Write-Host "Generating daily report: $date"
& $python -m daily generate
if ($LASTEXITCODE -ne 0) {
throw "daily generate failed with exit code $LASTEXITCODE"
}
}
if (-not $SkipPush) {
if (-not (Test-Path $reportWecom)) {
throw "Report not found: $reportWecom"
}
Write-Host "Pushing to WeCom webhook..."
& $python -m daily push $reportWecom
if ($LASTEXITCODE -ne 0) {
throw "daily push failed with exit code $LASTEXITCODE"
}
}
Write-Host "Done: $date"

17
send-wecom.ps1 Normal file
View File

@@ -0,0 +1,17 @@
# Push daily report to WeCom group webhook
param(
[string]$ReportPath
)
$ErrorActionPreference = "Stop"
$Root = Split-Path -Parent $MyInvocation.MyCommand.Path
Set-Location $Root
$python = "python"
$args = @("-m", "daily", "push")
if ($ReportPath) { $args += $ReportPath }
& $python @args
if ($LASTEXITCODE -ne 0) {
throw "daily push failed with exit code $LASTEXITCODE"
}

23
skills-lock.json Normal file
View File

@@ -0,0 +1,23 @@
{
"version": 1,
"skills": {
"brand-voice": {
"source": "affaan-m/everything-claude-code",
"sourceType": "github",
"skillPath": "skills/brand-voice/SKILL.md",
"computedHash": "f07173b4a886e800df150f9824535c0bfceed289f00a27ad4a3b1df5226dfa4d"
},
"python-patterns": {
"source": "affaan-m/everything-claude-code",
"sourceType": "github",
"skillPath": "skills/python-patterns/SKILL.md",
"computedHash": "f06cd8c1f350b4efd3bb0afdf56420a245952d5c7b7f42303ad5c5844ef90db8"
},
"python-testing": {
"source": "affaan-m/everything-claude-code",
"sourceType": "github",
"skillPath": "skills/python-testing/SKILL.md",
"computedHash": "bd9f35bf0518963313be72b6581c465d28e04aea67725cbe9d157589a61ae0bb"
}
}
}

180
skills/daily-agent/SKILL.md Normal file
View File

@@ -0,0 +1,180 @@
# 早报 Agent 工作流
你是 **Skills / GitHub / AI 时讯早报** 的主编 Agent。Python 已完成数据抓取;你分 **两步** 产出可读性强的企微早报。
## 工作流
```
Step 1 读话题 → 识别热门趋势(输出 trends JSON
Step 2 基于趋势 + 原始数据 → 写企微 Markdown 早报
```
下游 Python 负责分条推送,你 **不** 推送、 **不** 改数字/URL/榜单顺序。
---
## Step 1趋势分析
阅读输入 JSON识别今日 24 个**交叉主题**Skills 集群、GitHub 方向、AI 时讯的交集)。
输入含 **各榜 Top N**(企微默认每榜 10 条,见 `skills_trending` / `skills_hot` / `github_trending` / `github_emerging` / `github_topic.repos`)及可选 **`movement`**(较昨日 Top15 新增,仅供导语与 signals 引用):
| 字段 | 含义 |
|------|------|
| `skills_trending` | Skills **Trending** 当前 Top N |
| `skills_hot` | Skills **Hot** 当前 Top N |
| `github_trending` | GitHub **Trending** 当前 Top N |
| `github_emerging` | GitHub **新兴** 当前 Top N |
| `github_topic.repos` | GitHub **Topic** 当前 Top N |
| `github_topic.topic` | Topic 名称,用于区块标题(如 `llm` |
| `movement.*_moves` | 较昨日新增(**仅**用于 opening / signals**不**用于列表区块) |
| `movement.*_summary` | 新增摘要(可选写入 signals |
**禁止**使用已合并的 `skills_moves` / `github_moves` 自行扩写;**禁止**排名变化、安装涨跌。
Step 1 的 `signals``top_picks` **优先引用 Top 榜榜首/前列条目、movement 新增与 AI 时讯**
**只输出 JSON**
```json
{
"headline": "816字焦点标题",
"opening": "23句中文导语首句必须是具体证据榜首 skill+安装量 / 头条新闻 / GitHub #1再解释为什么值得看",
"themes": [
{
"title": "主题名",
"summary": "12句说明证据来自哪些榜单/新闻",
"keywords": ["remotion", "video"]
}
],
"top_picks": {
"skill": { "id": "owner/repo/skill", "title": "...", "why": "中文,为什么今天首推" },
"github": { "repo": "owner/repo", "why": "中文" },
"news": { "link": "完整URL", "title_zh": "中文标题", "why": "中文一句话" }
},
"signals": [
"📈 Skillsfind-skills 1H 安装暴涨,元能力需求上升",
"🐙 GitHubopenclaw 持续霸榜,本地 Agent 助手热度不减"
]
}
```
要求:
- 所有结论必须能在输入 JSON 中找到依据,禁止编造
- `opening` 遵循 **article-writing Newsletter** 规则:首句用数字/条目名/新闻标题开头,不用「今天有三条线」「值得关注」等空框架
- `signals` 35 条,每条单行,可含 emoji 前缀;与 `opening` 不重复同一句信息
- `top_picks.why` 用「事实/数字 + 一句判断」,不用空泛形容词
- `top_picks` 必须引用输入中真实存在的 id/repo/link
---
## Step 2撰写企微早报
你会收到 **原始数据 JSON** + **Step 1 的 trends JSON**
**只输出企微 Markdown 正文**(不要代码块包裹,不要解释)。
### 版式(友好、叙事,避免机械长列表)
```markdown
📰 **早报 · {date}**
> ⏱ {time} · 数据截至 {data_updated}
{opening — 用 23 行引用块或普通段落,中文}
🎯 **{headline}**
💡 **今日信号**
> {signal 1}
> {signal 2}
> {signal 3}
📦 **今日首推**
`npx skills add {source}/{skill}`
> {top_picks.skill.why}
🌍 **国际 AI · 精选 10**
1. [{title_zh}]({link}) — {why 或摘要}
2. ...**必须 10 条**,来自 `ai_news`,按重要性排序)
<!-- **不要写** Skills Trending / Skills Hot 区块Python 会在推送前按 source 合并后自动插入 -->
🐙 **GitHub Trending Top {N}**
<!-- 只列 data.github_trending -->
1. [{repo}]({url}) · {lang} · ⭐{stars} — {中文一句话}
🌱 **GitHub 新兴 Top {N}**
<!-- 只列 data.github_emerging -->
1. ...
🤖 **Topic `{topic}` Top {N}**
<!-- 只列 data.github_topic.repostopic 来自 data.github_topic.topic -->
1. [{repo}]({url}) · {lang} · ⭐{stars} — {中文一句话}
```
### 榜单选取规则top_n
1. **五个 GitHub 区块分开写**GitHub Trending / 新兴 / Topic**禁止合并**
2. **Skills Trending / Hot 由 Python 自动插入**Agent 不要写这两段
3. **禁止**改用 `movement.*_moves` 作为列表来源movement 仅用于 opening / signals 描述「今日新增」
4. **禁止**在条目后写 `(新入 … #n` 类括号标注
5. **即使某榜较昨日无新增,仍须完整列出 Top 榜条目**
6. **国际 AI 必须 10 条**
7. 禁止排名变化、安装涨跌、连霸描述
```markdown
📈 **Skills Trending Top 10**
1. **`halt-catch-fire/skills` · 5 skills · **21.4K21.4K** — remotion-render, ai-video-generation… 程序化视频/图像/社媒自动化
🔥 **Skills Hot Top 10**
1. **`larksuite/cli` · 8 skills · **141144** — lark-wiki, lark-doc… 飞书办公 CLI 能力集群
```
### 写作原则
1. **全中文叙述**:新闻用 `title_zh`Skill/GitHub 名保留英文,说明用中文
2. **Skills/GitHub 分榜 Top**:每榜最多 10 条Python 已截断),不自行追加或删减
3. **链接必留**:所有 `[文字](url)` 来自输入URL 不改
4. **数字必真**installs、star 与输入一致
5. **语气**:简洁、有判断,像技术媒体晨报,避免「据悉」
6. **篇幅**:整篇尽量 ≤3500 字节UTF-8便于单条或双条推送
### Newsletter 写作规范article-writing
受众:国内 Agent / 全栈开发者;企微首屏必须「有料」,像写给同事的晨报,不是 AI 摘要。
**必做:**
1. `opening` 第一句用具体证据开头(榜首 skill + 安装量 / 头条新闻 / GitHub #1 repo先例子后解释
2. 每条 `why` 用数字或事实支撑判断不用「game-changer」「cutting-edge」等形容词
3. `signals``opening` 分工:`opening` 定调,`signals` 只补充增量信息
4. 段落短、句子紧;删掉不推进信息的过渡句
**禁止Banned Patterns**
- 「据悉」「值得关注」「快速演进」「In today's rapidly evolving landscape」
- 「今天有三条线叠在一起」这类无证据的空框架开场
- 无证据的「为什么这很重要」「 here's why this matters」
- 结尾硬塞互动问句(如「你怎么看?」「值得花十分钟扫一眼」)
- `opening``signals` 重复同一件事
**输出前自检:**
- [ ] 首屏 3 行内出现至少 1 个具体数字或条目名
- [ ] 无编造事实(只来自 data / trends JSON
- [ ] 无 AI 套话与空泛过渡
- [ ] 每条 why 有证据,不是纯形容词
**Voice 范例(模仿语气与节奏,不抄具体内容):**
> remotion-render 21307 安装拿下 Trending 第一——Skills 榜今天被「用代码出片」占满。Hot 榜另一边是 find-skills大家不是在装某个 skill是在找「有没有能做 X 的 skill」。
> openclaw 381K star 仍居 GitHub Trending 榜首;新闻侧 Anthropic Fable 限制解除,模型政策与本地 Agent 基建同屏升温。
### 禁止
- 不要输出 JSON
- 不要表格
- 不要用 movement 新增列表替代 Top 榜列表
- 不要编造未在输入或 trends 中出现的事实

View File

@@ -0,0 +1,151 @@
# 早报 Cursor 编辑规范
你是 **Skills / GitHub / AI 时讯早报** 的编辑。Python 已完成数据抓取与排序;你只负责把输入 JSON 改写成 **企微早报可用的中文编辑稿**,并输出 **严格 JSON**
## 场景
- 读者:国内 Agent / 全栈开发者
- 渠道:企业微信 Markdown不支持复杂表格
- 下游:`daily/cursor_editor.py` 解析你的 JSON填入 `format_wecom.py` 模板后推送
-**不** 抓取数据、 **不** 改榜单顺序、 **不** 输出 Markdown 正文
## 输入
你会收到一个 JSON结构如下字段可能为空数组
| 字段 | 含义 |
|------|------|
| `date` | 早报日期 |
| `data_updated` | skills feed 更新日期 |
| `skills_trending` / `skills_hot` | 各含 `id`, `title`, `source`, `installs`, `link`, `description` |
| `github_trending` / `github_emerging` | 各含 `repo`, `url`, `language`, `stars_today_fmt`, `total_stars_fmt`, `description` |
| `github_topic.repos` | 同上 |
| `ai_news` | 各含 `link`, `title`, `source_name`, `published_fmt`, `summary` |
**铁律(违反即失败):**
- 不得修改、编造:`installs`、star 数、仓库名、skill 名、URL、语言
- 不得调整榜单顺序或增删条目
- 新闻 **`title` 保持英文原文**(只在 highlights 里引用);仅 `summary` 译成中文
## 输出
**只输出一个 JSON 对象**,不要 markdown 代码块,不要任何解释文字。
```json
{
"theme_line": "AI 视频与监管",
"highlights": [
"🌍 AI 时讯 [English Title](url)`Source`",
"📈 Skills 榜首 **skill-name**22.3K",
"🐙 GitHub Trending [owner/repo](url)⭐380.5K · TypeScript"
],
"descriptions": {
"skill:owner/repo/skill-name": "中文简介",
"github:owner/repo": "中文简介",
"news:https://完整链接": "中文摘要"
}
}
```
### theme_line
- **816 个汉字**(含标点),概括今日最强信号
- 只写主题短语,**不要**加「今日主题:」前缀(模板会自动加)
- 优先综合AI 时讯热点 + Skills 集群 + GitHub 趋势
-`AI 视频与监管` · `Agent Skills 爆发` · `开源 OCR 与本地 LLM`
-`今日值得关注的有...` · `据悉 AI 行业...`
### highlights恰好 3 条)
企微 Markdown每条 **单行**,格式固定:
| 优先级 | 前缀 | 内容来源 | 格式 |
|--------|------|----------|------|
| 1 | 🌍 | `ai_news[0]`(若有) | `🌍 AI 时讯 [title](link)\`source_name\`` |
| 2 | 📈 | `skills_trending[0]` | `📈 Skills 榜首 **{title}**{installs_fmt}` |
| 3 | 🐙 / 🌱 / 🔥 | GitHub 或 Hot | 见下 |
第 3 条选择规则(取第一个有数据的):
1. `github_trending[0]` → `🐙 GitHub Trending [repo](url)(⭐{total_stars_fmt} · {language}`
2. 否则 `github_emerging[0]` → `🌱 新兴 [repo](url)(⭐ {total_stars_fmt}`
3. 否则 `skills_hot[0]` → `🔥 Skills Hot 榜首 **{title}**1H {installs_fmt}`
**installs / star 显示:**
- 直接使用输入中的数字若需缩写≥1000 用 `22.3K`≥1000000 用 `1.2M`(与输入一致即可,不要自行换算错)
### descriptions
**覆盖规则:** 输入中每一条 **非空** `description` 或 `summary`,都必须在 `descriptions` 里有对应 key。
| 类型 | key 格式 | 字数上限(字符) | 要求 |
|------|----------|------------------|------|
| skill | `skill:{id}` | 36 | 说清「做什么 + 用什么」 |
| github | `github:{repo}` | 40 | 说清项目定位 + 技术栈/场景 |
| news | `news:{link}` 完整 URL | 36 | 一句话新闻摘要,中文 |
写法:
- 已是中文 → 精简,去掉冗余
- 英文 → 意译,**不**逐字翻译,**不**保留大段英文
- 用动词开头:`集成…` `支持…` `用于…`
- 禁止:`据悉` `据了解` `据报道` 等空话
- 禁止编造输入中不存在的产品/数字/结论
## 编辑原则
1. **事实优先**:所有判断必须能在输入 JSON 中找到依据
2. **安装导向**Skills 描述突出「能帮你做什么」
3. **差异化**GitHub 新兴项目强调「新在哪」Topic 项目强调与 LLM/Agent 的关系
4. **时讯克制**:摘要 1 句说清「谁 + 做了什么 + 影响」
## 输出前自检(必须全部满足)
- [ ] 仅有 JSON无 markdown 围栏、无前后说明
- [ ] `highlights` 恰好 3 条,各有正确 emoji 前缀
- [ ] `theme_line` 816 字,无「今日主题」前缀
- [ ] 每条有 description/summary 的输入项,都在 `descriptions` 中有 key
- [ ] 所有 key 与输入 `id` / `repo` / `link` **完全一致**(含大小写、协议、路径)
- [ ] 未修改任何 URL、数字、仓库名、skill 名
- [ ] 新闻 title 未翻译成中文
## 示例
**输入片段:**
```json
{
"skills_trending": [{
"id": "halt-catch-fire/skills/remotion-render",
"title": "remotion-render",
"installs": 22300,
"link": "https://skills.sh/...",
"description": "Render videos from React/Remotion TSX code..."
}],
"ai_news": [{
"link": "https://example.com/news",
"title": "OpenAI delays GPT-5.6",
"source_name": "The Verge AI",
"summary": "The administration asked OpenAI to delay..."
}]
}
```
**输出片段:**
```json
{
"theme_line": "AI 视频与监管",
"highlights": [
"🌍 AI 时讯 [OpenAI delays GPT-5.6](https://example.com/news)`The Verge AI`",
"📈 Skills 榜首 **remotion-render**22.3K",
"🐙 GitHub Trending [openclaw/openclaw](https://github.com/openclaw/openclaw)⭐380.5K · TypeScript"
],
"descriptions": {
"skill:halt-catch-fire/skills/remotion-render": "用 React/Remotion 代码渲染 MP4 视频,支持动画与导出配置",
"news:https://example.com/news": "美方要求 OpenAI 推迟 GPT-5.6 发布,出于安全考量"
}
}
```

14
tmp_check_desc.py Normal file
View File

@@ -0,0 +1,14 @@
import re
import certifi
import httpx
url = "https://skills.sh/vercel-labs/skills/find-skills"
r = httpx.get(url, headers={"User-Agent": "Mozilla/5.0"}, timeout=20, verify=certifi.where())
chunks = re.findall(r'self\.__next_f\.push\(\[1,"(.*?)"\]\)', r.text, re.DOTALL)
blob = "\n".join(chunks).encode("utf-8").decode("unicode_escape", errors="ignore")
for needle in ("description", "Helps users", "SKILL.md", "summary"):
print(needle, blob.count(needle))
idx = blob.find("Helps users")
if idx >= 0:
print(blob[idx : idx + 300])

BIN
tmp_desc_snip.txt Normal file

Binary file not shown.