Compare commits
3 Commits
b7f4bbe03c
...
89a48a98aa
| Author | SHA1 | Date | |
|---|---|---|---|
| 89a48a98aa | |||
| c780b269d6 | |||
| d4590fbb86 |
78
docs/实施方案.md
78
docs/实施方案.md
@@ -143,7 +143,7 @@ flowchart TB
|
||||
1. **纯静态分析**:基于 Java 源码 AST + 符号解析,不启动 Spring 容器
|
||||
2. **Diff 驱动**:只分析本次 push 变更涉及的文件及其关联类型
|
||||
3. **本仓限定**:类型解析仅在业务仓库 `src/main/java` 范围内
|
||||
4. **可配置**:忽略规则、严重级别、通知开关、阻断开关均可 YAML 配置
|
||||
4. **可配置**:忽略规则、通知开关、阻断开关均可 YAML 配置
|
||||
5. **可演进**:已覆盖 JSON 字符串写入与 Template 直写 / Hash;后续可扩展读路径反向确认、报告落盘等
|
||||
|
||||
---
|
||||
@@ -163,7 +163,7 @@ flowchart TB
|
||||
|
||||
**不采用** Spoon / Eclipse JDT 的原因:JavaParser 足够覆盖第一版需求,依赖更轻,CLI 启动更快。
|
||||
|
||||
**Lombok 处理策略**:第一版基于源码字段 + `@Data` 等注解推断序列化字段;对 `@Builder`、`@SuperBuilder` 等复杂场景标记为「低置信度」并降级为 P2 提示。后续可选集成 `lombok.ast` 或 delombok 预处理。
|
||||
**Lombok 处理策略**:基于源码字段 + `@Data` 等注解推断序列化字段;对 `@Builder`、`@SuperBuilder` 等复杂场景标记为低置信度提示。后续可选集成 `lombok.ast` 或 delombok 预处理。
|
||||
|
||||
---
|
||||
|
||||
@@ -282,9 +282,8 @@ git diff --name-only {old-sha} {new-sha} -- '*.java'
|
||||
| W01 | `redisUtil.insert(key, JSON.toJSONString(expr), ttl)` | key 表达式、value 表达式 | ✅ |
|
||||
| W02 | `redisTemplate.opsForValue().set(key, JSON.toJSONString(expr), ...)` | 同上 | ✅ |
|
||||
| W03 | `stringRedisTemplate.opsForValue().set(key, JsonUtil.getObjectToString(expr), ...)` | 同上 | ✅ |
|
||||
| W04 | `redisTemplate.opsForValue().set(key, expr, ...)` 且 expr 非字面量 | 直写对象类型 | ✅ Phase 2 |
|
||||
| W05 | `redisTemplate.opsForHash().put(key, field, expr)` | Hash 写出 value 类型 | ✅ Phase 2 |
|
||||
| W06 | `JSON.parseObject(cacheValue, Xxx.class)` | 辅助反向确认读取类型 | 未做 |
|
||||
| W04 | `redisTemplate.opsForValue().set(key, expr, ...)` 且 expr 非字面量 | 直写对象类型 | ✅ |
|
||||
| W05 | `redisTemplate.opsForHash().put(key, field, expr)` | Hash 写出 value 类型 | ✅ |
|
||||
|
||||
**忽略规则**(自动):
|
||||
|
||||
@@ -292,6 +291,19 @@ git diff --name-only {old-sha} {new-sha} -- '*.java'
|
||||
- 方法名含 `setIfAbsent`、`increment`、`delete`、`remove`、`expire` 等
|
||||
- key 匹配 `ignore.key_patterns` 配置(锁 / token / 登录计数等)
|
||||
|
||||
#### Step 4b:读侧类型辅助(W06,非写入模式)
|
||||
|
||||
W06 **不产生独立告警**,只扫描反序列化调用,用读到的 `Xxx.class` **补强**同文件(或同 key)写入点的 value 类型 / 根数组标记。
|
||||
|
||||
| 匹配示例 | 作用 |
|
||||
|----------|------|
|
||||
| `JSON.parseObject(raw, Xxx.class)` | 补强对象类型 |
|
||||
| `JSON.parseArray(raw, Xxx.class)` / `JsonUtil.getJsonToList` | 补强 `List<Xxx>`(rootArray) |
|
||||
| `JsonUtil.getJsonToBean(raw, Xxx.class)` | 同上 |
|
||||
| 可关联到 `redisUtil.getString(key)` / `opsForValue().get(key)` | 同时补强 key 模式 |
|
||||
|
||||
开关:`detection.read_hints_enabled`(默认 `true`)。优先级:`manual_mappings` > W06 补强 > 写侧 AST 推断。
|
||||
|
||||
#### Step 5:类型推断
|
||||
|
||||
对每个写入点的 `expr`,使用 JavaParser Symbol Solver 推断类型:
|
||||
@@ -320,7 +332,7 @@ redisUtil.insert(buildCacheKey(encode), JSON.toJSONString(envelope), ttl);
|
||||
| `@JSONField(name = "xxx")` | 字段名映射 | ✅ |
|
||||
| `@JsonIgnore` | 排除字段 | ✅ |
|
||||
| `@JsonProperty("xxx")` | 字段名映射 | ✅ |
|
||||
| `@JsonIgnoreProperties({...})` | 类级忽略字段 | ✅ Phase 2 |
|
||||
| `@JsonIgnoreProperties({...})` | 类级忽略字段 | ✅ |
|
||||
| `@Schema` | 忽略(不影响序列化) | ✅ |
|
||||
|
||||
#### Step 6:生成 JSON Schema
|
||||
@@ -353,19 +365,19 @@ redisUtil.insert(buildCacheKey(encode), JSON.toJSONString(envelope), ttl);
|
||||
|
||||
对比同一写入点在 old/new 两个版本的 `TypeSchema`,输出 `SchemaChange` 列表。
|
||||
|
||||
**变更类型与严重级别**:
|
||||
**变更类型**(有结构差异即告警;`block` 下任意变更均阻断):
|
||||
|
||||
| 变更类型 | 示例 | 默认级别 |
|
||||
|----------|------|----------|
|
||||
| `FIELD_REMOVED` | 删除 `dbName` | P0 |
|
||||
| `TYPE_CHANGED` | `linkList` 从数组变对象 | P0 |
|
||||
| `WRAPPER_ADDED` | 顶层增加 `vo` 包装 | P0 |
|
||||
| `FIELD_PATH_MOVED` | `dbName` → `vo.dbName` | P0 |
|
||||
| `FIELD_ADDED` | 新增 `expiresAtMs` | P1 |
|
||||
| `KEY_PATTERN_CHANGED` | key 常量变更 | P1 |
|
||||
| `WRITE_POINT_REMOVED` | 删除缓存写入 | P1 |
|
||||
| `WRITE_POINT_ADDED` | 新增缓存写入 | P2 |
|
||||
| `LOW_CONFIDENCE` | 类型推断失败 | P2 |
|
||||
| 变更类型 | 示例 |
|
||||
|----------|------|
|
||||
| `FIELD_REMOVED` | 删除 `dbName` |
|
||||
| `TYPE_CHANGED` | `linkList` 从数组变对象 |
|
||||
| `WRAPPER_ADDED` | 顶层增加 `vo` 包装 |
|
||||
| `FIELD_PATH_MOVED` | `dbName` → `vo.dbName` |
|
||||
| `FIELD_ADDED` | 新增 `expiresAtMs` |
|
||||
| `KEY_PATTERN_CHANGED` | key 常量变更 |
|
||||
| `WRITE_POINT_REMOVED` | 删除缓存写入 |
|
||||
| `WRITE_POINT_ADDED` | 新增缓存写入 |
|
||||
| `LOW_CONFIDENCE` | 类型推断失败(仍提示,置信度较低) |
|
||||
|
||||
#### Step 8:报告与通知
|
||||
|
||||
@@ -377,13 +389,13 @@ redisUtil.insert(buildCacheKey(encode), JSON.toJSONString(envelope), ttl);
|
||||
|
||||
企微 Markdown 规则:
|
||||
|
||||
- 抬头不含 mode / P0~P2 汇总;正文按 Key 展示骨架
|
||||
- 抬头不含 mode 汇总;正文按 Key 展示骨架
|
||||
- **删除字段**:旧骨架中橙色 `<font color="warning">`
|
||||
- **新增字段**:新骨架中绿色 `<font color="info">`
|
||||
- key 未解析时展示源码表达式 + 灰色「(key 未解析)」
|
||||
- 单条超 4096 UTF-8 字节时按 Key 拆成多条依次发送
|
||||
|
||||
CI 控制台额外输出字段明细(含严重级别),再打印与企微一致的 Markdown。
|
||||
CI 控制台额外输出字段级变更明细(变更类型 + 位置 + 摘要),再打印与企微一致的 Markdown。
|
||||
|
||||
调用企微 Webhook 发送 Markdown(支持 `--dry-run` 仅本地输出)。
|
||||
|
||||
@@ -430,7 +442,7 @@ manual_mappings:
|
||||
|
||||
| 层级 | 位置 | 职责 |
|
||||
|------|------|------|
|
||||
| 默认配置 | 工具 jar 内 `default-config.yaml` | 检测模式、忽略规则、严重级别默认值 |
|
||||
| 默认配置 | 工具 jar 内 `default-config.yaml` | 检测模式、忽略规则等默认值 |
|
||||
| 业务覆盖 | `jnpf-java-cloud/.gitea/config/cache-schema-check-config.yaml` | mode、notify、include_modules、manual_mappings |
|
||||
|
||||
合并规则:**业务配置覆盖默认配置**,未声明的项沿用默认值。
|
||||
@@ -481,7 +493,7 @@ notify:
|
||||
|
||||
## 10. 分阶段交付计划
|
||||
|
||||
### Phase 1 — MVP(约 1.5 周)✅
|
||||
### Phase 1 — MVP✅
|
||||
|
||||
**目标**:跑通端到端链路,覆盖租户缓存典型场景。
|
||||
|
||||
@@ -491,10 +503,10 @@ notify:
|
||||
| Git diff 扫描 | 变更文件列表 | ✅ |
|
||||
| W01~W03 写入点检测 | 覆盖 JSON 字符串写入 | ✅ |
|
||||
| 基础 Schema 提取 | 支持普通类、内部类、List、嵌套 | ✅ |
|
||||
| Schema Diff P0/P1 | 字段增删、包装、路径迁移 | ✅ |
|
||||
| Schema Diff | 字段增删、包装、路径迁移 | ✅ |
|
||||
| 企微通知 | 按 Key 骨架 Markdown | ✅ |
|
||||
| notify/block / enabled | 配置驱动 | ✅ |
|
||||
| 夹具测试 | TenantVO/CacheEnvelope 样本 | ✅ |
|
||||
| 样本夹具测试 | TenantVO/CacheEnvelope 等 | ✅ |
|
||||
|
||||
**验收标准**:
|
||||
|
||||
@@ -502,7 +514,7 @@ notify:
|
||||
- 流水线 push 后能收到企微通知
|
||||
- `mode=block` 时任意结构变更导致 exit 1
|
||||
|
||||
### Phase 2 — 增强(约 1 周)✅
|
||||
### Phase 2 — 增强✅
|
||||
|
||||
| 任务 | 说明 | 状态 |
|
||||
|------|------|------|
|
||||
@@ -512,8 +524,9 @@ notify:
|
||||
| 忽略规则完善 | 锁/计数器/token/字面量/setIfAbsent | ✅ |
|
||||
| 多模块性能 | 并行读文件、索引批量装载、`manual_mappings` | ✅ |
|
||||
| 企微高亮 | 删除橙 `warning` / 新增绿 `info`;位置+类型通用项 | ✅ |
|
||||
| W06 读侧辅助 | `parseObject`/`parseArray` 等补强写入 value 类型 | ✅ |
|
||||
|
||||
### Phase 3 — 运营(约 0.5 周)
|
||||
### Phase 3 — 运营
|
||||
|
||||
| 任务 | 说明 |
|
||||
|------|------|
|
||||
@@ -532,12 +545,14 @@ notify:
|
||||
- `RedisWritePointDetectorTest`:各种写入 AST 模式匹配
|
||||
- `RedisKeyResolverTest`:常量、format、拼接推断
|
||||
|
||||
### 11.2 夹具集成测试
|
||||
### 11.2 样本夹具测试(fixtures)
|
||||
|
||||
在 `src/test/resources/fixtures/` 放置真实业务代码片段(从 `jnpf-java-cloud` 提取并脱敏),模拟 old/new 两个版本:
|
||||
「夹具」= 放在测试资源里的**脱敏源码样本**(不是连真实 Redis / 不是起 Gitea 流水线)。
|
||||
|
||||
| 夹具 | 验证点 |
|
||||
|------|--------|
|
||||
路径:`src/test/resources/fixtures/`。测试代码加载这些 `.txt`/Java 片段,在内存中跑检测器 / Schema 对比,用来验证典型业务场景是否被正确识别。
|
||||
|
||||
| 夹具目录 | 验证点 |
|
||||
|----------|--------|
|
||||
| `fixtures/tenant/` | 包装结构变更(TenantVO → CacheEnvelope) |
|
||||
| `fixtures/lock/` | 锁/计数器/token 应被忽略 |
|
||||
| `fixtures/template/` | W04 Template 直写 |
|
||||
@@ -569,7 +584,7 @@ notify:
|
||||
| 1 | 阻断范围 | `block` 模式下 **任意结构变更均阻断**(exit 1) |
|
||||
| 2 | 发布坐标 | 独立产物 `com.codechecker:cache-schema-checker:1.0.0` |
|
||||
| 3 | 配置归属 | **双层配置**:jar 内 `default-config.yaml` + 业务仓覆盖合并 |
|
||||
| 4 | 上线策略 | 先 `notify` 观察 **1 周**,稳定后手动切 `block` |
|
||||
| 4 | 上线策略 | 先 `notify` ,稳定后手动切 `block` |
|
||||
| 5 | 检测范围 | **仅 `src/main/java`**,不扫描测试代码 |
|
||||
|
||||
以上决策已纳入实施方案;**Phase 1 / Phase 2 已交付**,可进入 Phase 3 或业务仓全量观察。
|
||||
@@ -598,7 +613,6 @@ public class WritePoint {
|
||||
|
||||
```java
|
||||
public class SchemaChange {
|
||||
Severity severity; // P0, P1, P2
|
||||
ChangeType changeType; // FIELD_REMOVED, WRAPPER_ADDED, ...
|
||||
String keyPattern;
|
||||
String writeLocation; // class#method:line
|
||||
|
||||
54
docs/配置说明.md
54
docs/配置说明.md
@@ -43,6 +43,7 @@ include_modules:
|
||||
由 `redisCheck` 仓库维护,随 jar 发布,默认包含:
|
||||
|
||||
- `detection.patterns`:**W01~W05**(JSON 字符串写入 + Template 直写 + Hash)
|
||||
- `detection.read_hints_enabled`:W06 读侧反序列化类型辅助(默认 true)
|
||||
- `ignore.key_patterns`(锁 / 计数器 / token)
|
||||
- `detection.min_confidence`、`max_field_depth`
|
||||
- `mode: notify`、`enabled: true`
|
||||
@@ -106,17 +107,15 @@ detection:
|
||||
- W04 # redisTemplate 直写对象
|
||||
- W05 # opsForHash().put
|
||||
|
||||
# 类型推断最低置信度,低于此值仅输出 P2 提示
|
||||
# W06:读侧反序列化类型辅助(不产生独立告警)
|
||||
read_hints_enabled: true
|
||||
|
||||
# 类型推断最低置信度,低于此值标记为低置信度提示
|
||||
min_confidence: 0.6
|
||||
|
||||
# 字段展开最大深度(防止循环引用死循环)
|
||||
max_field_depth: 8
|
||||
|
||||
# 严重级别覆盖(可选)
|
||||
severity_overrides:
|
||||
FIELD_ADDED: P1
|
||||
WRITE_POINT_ADDED: P2
|
||||
|
||||
# 人工补充映射(自动推断失败或需精确指定时使用)
|
||||
manual_mappings:
|
||||
- id: tenant-db-content
|
||||
@@ -216,6 +215,23 @@ detection:
|
||||
patterns: [W01, W02, W03]
|
||||
```
|
||||
|
||||
### 3.5.1 detection.read_hints_enabled(W06 辅助)
|
||||
|
||||
| 字段 | 类型 | 默认 | 说明 |
|
||||
|------|------|------|------|
|
||||
| `read_hints_enabled` | boolean | true | 扫描 `parseObject` / `parseArray` / `getJsonToBean` 等,补强同文件或同 key 写入点的 value 类型 |
|
||||
|
||||
- **不是写入模式**:不会单独因为「多了一处 parse」而告警
|
||||
- 能关联到 `redis get(key)` 时,还可补强 unresolved key
|
||||
- 覆盖优先级:`manual_mappings` > W06 > 写侧 AST
|
||||
|
||||
关闭示例:
|
||||
|
||||
```yaml
|
||||
detection:
|
||||
read_hints_enabled: false
|
||||
```
|
||||
|
||||
### 3.6 manual_mappings
|
||||
|
||||
当自动推断不准确时使用。匹配优先级 **高于** 自动推断(按 `类全名#方法名` 覆盖 key 模式与 value 类型)。
|
||||
@@ -266,10 +282,10 @@ suppressions:
|
||||
|
||||
### 5.1 结构说明
|
||||
|
||||
- 抬头:仓库、分支、提交、提交人、时间(**不再**展示 mode / P0P1P2 汇总)
|
||||
- 抬头:仓库、分支、提交、提交人、时间(不展示 mode)
|
||||
- 正文:按 **一个 Redis Key 一块**,展示位置、类型与前后序列化骨架
|
||||
- 超长(UTF-8 > 4096 字节)时按 key **拆成多条**消息依次发送
|
||||
- CI 控制台另打「字段明细」(含 P0/P1/P2),企微侧不分级别
|
||||
- CI 控制台另打「字段明细」(变更类型 / 位置 / 摘要),企微侧按骨架展示
|
||||
|
||||
### 5.2 字段高亮颜色
|
||||
|
||||
@@ -315,8 +331,6 @@ suppressions:
|
||||
|
||||
## 6. 推荐上线配置
|
||||
|
||||
### 6.1 观察期(第 1 周)
|
||||
|
||||
业务仓默认配置:
|
||||
|
||||
```yaml
|
||||
@@ -327,23 +341,5 @@ notify:
|
||||
enabled: true
|
||||
webhook_url: "" # 由流水线写入真实 Webhook
|
||||
|
||||
include_modules:
|
||||
- jnpf-tenant
|
||||
```
|
||||
|
||||
观察满 1 周、确认误报可接受后,手动切换:
|
||||
|
||||
```yaml
|
||||
mode: block
|
||||
include_modules: [] # 扩至全仓
|
||||
```
|
||||
|
||||
### 6.2 全量启用(观察期结束后)
|
||||
|
||||
```yaml
|
||||
enabled: true
|
||||
mode: block
|
||||
include_modules: [] # 空表示全部模块
|
||||
detection:
|
||||
patterns: [W01, W02, W03, W04, W05]
|
||||
include_modules: [] # 全仓
|
||||
```
|
||||
@@ -1,6 +1,8 @@
|
||||
package com.codechecker.cache.analyze;
|
||||
|
||||
import com.codechecker.cache.config.CheckerConfig;
|
||||
import com.codechecker.cache.detector.CacheReadHint;
|
||||
import com.codechecker.cache.detector.CacheReadHintDetector;
|
||||
import com.codechecker.cache.detector.RedisWritePointDetector;
|
||||
import com.codechecker.cache.detector.WritePoint;
|
||||
import com.codechecker.cache.diff.ChangeType;
|
||||
@@ -111,6 +113,10 @@ public class SchemaCheckAnalyzer {
|
||||
List<WritePoint> newWps = detectorNew.detect(path, newContent);
|
||||
List<WritePoint> oldWps = oldContent == null
|
||||
? new ArrayList<>() : detectorOld.detect(path, oldContent);
|
||||
applyReadHints(newWps, path, newContent, newIndex);
|
||||
if (oldContent != null) {
|
||||
applyReadHints(oldWps, path, oldContent, oldIndex);
|
||||
}
|
||||
newWps.forEach(this::applyManualMappings);
|
||||
oldWps.forEach(this::applyManualMappings);
|
||||
|
||||
@@ -425,6 +431,92 @@ public class SchemaCheckAnalyzer {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* W06:用同文件读侧反序列化类型提示,补强低置信度 / 缺类型的写入点。
|
||||
* manual_mappings 仍在其后执行,可覆盖本补强结果。
|
||||
*/
|
||||
private void applyReadHints(List<WritePoint> writePoints, String path, String content,
|
||||
SourceIndex index) {
|
||||
if (!config.getDetection().isReadHintsEnabled()
|
||||
|| writePoints == null || writePoints.isEmpty()) {
|
||||
return;
|
||||
}
|
||||
List<CacheReadHint> hints = new CacheReadHintDetector(index).detect(path, content);
|
||||
if (hints.isEmpty()) {
|
||||
return;
|
||||
}
|
||||
for (WritePoint wp : writePoints) {
|
||||
enrichWritePointFromHints(wp, hints);
|
||||
}
|
||||
}
|
||||
|
||||
private void enrichWritePointFromHints(WritePoint wp, List<CacheReadHint> hints) {
|
||||
boolean needType = wp.getResolvedValueType() == null || wp.getResolvedValueType().isEmpty()
|
||||
|| wp.getConfidence() < config.getDetection().getMinConfidence();
|
||||
boolean needKey = isUnresolvedKey(wp.getResolvedKeyPattern());
|
||||
if (!needType && !needKey) {
|
||||
return;
|
||||
}
|
||||
|
||||
CacheReadHint best = null;
|
||||
int bestScore = -1;
|
||||
for (CacheReadHint hint : hints) {
|
||||
int score = scoreHint(wp, hint);
|
||||
if (score > bestScore) {
|
||||
bestScore = score;
|
||||
best = hint;
|
||||
}
|
||||
}
|
||||
if (best == null || bestScore <= 0) {
|
||||
return;
|
||||
}
|
||||
if (needType && best.getResolvedValueType() != null) {
|
||||
wp.setResolvedValueType(best.getResolvedValueType());
|
||||
wp.setRootArray(best.isRootArray());
|
||||
wp.setConfidence(Math.max(wp.getConfidence(), best.getConfidence()));
|
||||
}
|
||||
if (needKey && best.getResolvedKeyPattern() != null
|
||||
&& !isUnresolvedKey(best.getResolvedKeyPattern())) {
|
||||
wp.setResolvedKeyPattern(best.getResolvedKeyPattern());
|
||||
if (best.getKeyExpression() != null) {
|
||||
// 保留写入侧原始表达式,仅补 key 模式
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** 匹配得分:同 key > 同类同方法 > 同类;无交集则 0。 */
|
||||
private int scoreHint(WritePoint wp, CacheReadHint hint) {
|
||||
if (hint.getResolvedValueType() == null) {
|
||||
return 0;
|
||||
}
|
||||
boolean sameClass = wp.getEnclosingClass() != null
|
||||
&& wp.getEnclosingClass().equals(hint.getEnclosingClass());
|
||||
if (!sameClass) {
|
||||
// 跨文件仅允许 key 模式已解析且一致
|
||||
if (wp.getResolvedKeyPattern() != null
|
||||
&& wp.getResolvedKeyPattern().equals(hint.getResolvedKeyPattern())
|
||||
&& !isUnresolvedKey(wp.getResolvedKeyPattern())) {
|
||||
return 40;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
int score = 10;
|
||||
if (wp.getEnclosingMethod() != null
|
||||
&& wp.getEnclosingMethod().equals(hint.getEnclosingMethod())) {
|
||||
score += 20;
|
||||
}
|
||||
if (wp.getResolvedKeyPattern() != null
|
||||
&& wp.getResolvedKeyPattern().equals(hint.getResolvedKeyPattern())
|
||||
&& !isUnresolvedKey(wp.getResolvedKeyPattern())) {
|
||||
score += 50;
|
||||
} else if (hint.getResolvedKeyPattern() != null
|
||||
&& !isUnresolvedKey(hint.getResolvedKeyPattern())
|
||||
&& isUnresolvedKey(wp.getResolvedKeyPattern())) {
|
||||
score += 30;
|
||||
}
|
||||
return score;
|
||||
}
|
||||
|
||||
private void collectTypeNames(String content, Set<String> out) {
|
||||
if (content == null || content.isEmpty()) {
|
||||
return;
|
||||
|
||||
@@ -110,6 +110,8 @@ public class CheckerConfig {
|
||||
private List<String> patterns = new ArrayList<>();
|
||||
private double minConfidence = 0.6;
|
||||
private int maxFieldDepth = 8;
|
||||
/** W06:是否启用读侧反序列化类型辅助补强 */
|
||||
private boolean readHintsEnabled = true;
|
||||
|
||||
public List<String> getPatterns() {
|
||||
return patterns;
|
||||
@@ -134,6 +136,14 @@ public class CheckerConfig {
|
||||
public void setMaxFieldDepth(int maxFieldDepth) {
|
||||
this.maxFieldDepth = maxFieldDepth;
|
||||
}
|
||||
|
||||
public boolean isReadHintsEnabled() {
|
||||
return readHintsEnabled;
|
||||
}
|
||||
|
||||
public void setReadHintsEnabled(boolean readHintsEnabled) {
|
||||
this.readHintsEnabled = readHintsEnabled;
|
||||
}
|
||||
}
|
||||
|
||||
public static class ManualMapping {
|
||||
|
||||
@@ -100,6 +100,7 @@ public final class ConfigLoader {
|
||||
d.setPatterns(strList(detection.get("patterns")));
|
||||
d.setMinConfidence(dbl(detection, "min_confidence", 0.6));
|
||||
d.setMaxFieldDepth((int) lng(detection, "max_field_depth", 8));
|
||||
d.setReadHintsEnabled(bool(detection, "read_hints_enabled", true));
|
||||
|
||||
Map<String, Object> severityOverrides = asMap(map.get("severity_overrides"));
|
||||
Map<String, String> so = new LinkedHashMap<>();
|
||||
|
||||
92
src/main/java/com/codechecker/cache/detector/CacheReadHint.java
vendored
Normal file
92
src/main/java/com/codechecker/cache/detector/CacheReadHint.java
vendored
Normal file
@@ -0,0 +1,92 @@
|
||||
package com.codechecker.cache.detector;
|
||||
|
||||
/**
|
||||
* 读侧反序列化类型提示(W06):从 parseObject / getJsonToBean 等推断「缓存 value 被当成什么类型用」。
|
||||
* 不单独产生告警,仅用于补强同文件/同 key 写入点的 value 类型。
|
||||
*/
|
||||
public class CacheReadHint {
|
||||
|
||||
private String filePath;
|
||||
private int lineNumber;
|
||||
private String enclosingClass;
|
||||
private String enclosingMethod;
|
||||
/** 推断出的 key 模式;无法关联 redis get 时为 null */
|
||||
private String resolvedKeyPattern;
|
||||
private String keyExpression;
|
||||
/** value 元素/对象 FQN */
|
||||
private String resolvedValueType;
|
||||
private boolean rootArray;
|
||||
private double confidence = 0.7;
|
||||
|
||||
public String getFilePath() {
|
||||
return filePath;
|
||||
}
|
||||
|
||||
public void setFilePath(String filePath) {
|
||||
this.filePath = filePath;
|
||||
}
|
||||
|
||||
public int getLineNumber() {
|
||||
return lineNumber;
|
||||
}
|
||||
|
||||
public void setLineNumber(int lineNumber) {
|
||||
this.lineNumber = lineNumber;
|
||||
}
|
||||
|
||||
public String getEnclosingClass() {
|
||||
return enclosingClass;
|
||||
}
|
||||
|
||||
public void setEnclosingClass(String enclosingClass) {
|
||||
this.enclosingClass = enclosingClass;
|
||||
}
|
||||
|
||||
public String getEnclosingMethod() {
|
||||
return enclosingMethod;
|
||||
}
|
||||
|
||||
public void setEnclosingMethod(String enclosingMethod) {
|
||||
this.enclosingMethod = enclosingMethod;
|
||||
}
|
||||
|
||||
public String getResolvedKeyPattern() {
|
||||
return resolvedKeyPattern;
|
||||
}
|
||||
|
||||
public void setResolvedKeyPattern(String resolvedKeyPattern) {
|
||||
this.resolvedKeyPattern = resolvedKeyPattern;
|
||||
}
|
||||
|
||||
public String getKeyExpression() {
|
||||
return keyExpression;
|
||||
}
|
||||
|
||||
public void setKeyExpression(String keyExpression) {
|
||||
this.keyExpression = keyExpression;
|
||||
}
|
||||
|
||||
public String getResolvedValueType() {
|
||||
return resolvedValueType;
|
||||
}
|
||||
|
||||
public void setResolvedValueType(String resolvedValueType) {
|
||||
this.resolvedValueType = resolvedValueType;
|
||||
}
|
||||
|
||||
public boolean isRootArray() {
|
||||
return rootArray;
|
||||
}
|
||||
|
||||
public void setRootArray(boolean rootArray) {
|
||||
this.rootArray = rootArray;
|
||||
}
|
||||
|
||||
public double getConfidence() {
|
||||
return confidence;
|
||||
}
|
||||
|
||||
public void setConfidence(double confidence) {
|
||||
this.confidence = confidence;
|
||||
}
|
||||
}
|
||||
189
src/main/java/com/codechecker/cache/detector/CacheReadHintDetector.java
vendored
Normal file
189
src/main/java/com/codechecker/cache/detector/CacheReadHintDetector.java
vendored
Normal file
@@ -0,0 +1,189 @@
|
||||
package com.codechecker.cache.detector;
|
||||
|
||||
import com.codechecker.cache.key.RedisKeyResolver;
|
||||
import com.codechecker.cache.schema.SourceIndex;
|
||||
import com.github.javaparser.StaticJavaParser;
|
||||
import com.github.javaparser.ast.CompilationUnit;
|
||||
import com.github.javaparser.ast.body.CallableDeclaration;
|
||||
import com.github.javaparser.ast.body.ClassOrInterfaceDeclaration;
|
||||
import com.github.javaparser.ast.body.VariableDeclarator;
|
||||
import com.github.javaparser.ast.expr.ClassExpr;
|
||||
import com.github.javaparser.ast.expr.Expression;
|
||||
import com.github.javaparser.ast.expr.MethodCallExpr;
|
||||
import com.github.javaparser.ast.expr.NameExpr;
|
||||
import com.github.javaparser.ast.type.ClassOrInterfaceType;
|
||||
import com.github.javaparser.ast.type.Type;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.Arrays;
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.Optional;
|
||||
import java.util.Set;
|
||||
|
||||
/**
|
||||
* W06:扫描读侧反序列化,提取「缓存字符串 → 业务类型」提示,供写入点类型补强。
|
||||
* <p>典型模式:</p>
|
||||
* <pre>
|
||||
* String raw = redisUtil.getString(key);
|
||||
* Foo vo = JSON.parseObject(raw, Foo.class);
|
||||
* List<Foo> list = JSON.parseArray(raw, Foo.class);
|
||||
* </pre>
|
||||
*/
|
||||
public class CacheReadHintDetector {
|
||||
|
||||
private static final Set<String> OBJECT_PARSE = new HashSet<>(Arrays.asList(
|
||||
"parseObject", "parse", "getJsonToBean", "toJavaObject", "readValue"));
|
||||
private static final Set<String> ARRAY_PARSE = new HashSet<>(Arrays.asList(
|
||||
"parseArray", "getJsonToList", "parseArrayObject"));
|
||||
private static final Set<String> REDIS_GET = new HashSet<>(Arrays.asList(
|
||||
"get", "getString", "opsForValue"));
|
||||
|
||||
private final SourceIndex index;
|
||||
private final RedisKeyResolver keyResolver;
|
||||
|
||||
public CacheReadHintDetector(SourceIndex index) {
|
||||
this.index = index;
|
||||
this.keyResolver = new RedisKeyResolver(index);
|
||||
}
|
||||
|
||||
public List<CacheReadHint> detect(String filePath, String content) {
|
||||
List<CacheReadHint> result = new ArrayList<>();
|
||||
if (content == null || content.isEmpty()) {
|
||||
return result;
|
||||
}
|
||||
CompilationUnit cu;
|
||||
try {
|
||||
cu = StaticJavaParser.parse(content);
|
||||
} catch (RuntimeException e) {
|
||||
return result;
|
||||
}
|
||||
|
||||
for (MethodCallExpr mce : cu.findAll(MethodCallExpr.class)) {
|
||||
CacheReadHint hint = tryParseHint(mce, filePath);
|
||||
if (hint != null) {
|
||||
result.add(hint);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private CacheReadHint tryParseHint(MethodCallExpr mce, String filePath) {
|
||||
String name = mce.getNameAsString();
|
||||
boolean array = ARRAY_PARSE.contains(name);
|
||||
boolean object = OBJECT_PARSE.contains(name);
|
||||
if (!array && !object) {
|
||||
return null;
|
||||
}
|
||||
if (mce.getArguments().size() < 2) {
|
||||
return null;
|
||||
}
|
||||
|
||||
Expression classArg = mce.getArgument(1);
|
||||
// readValue(str, TypeReference) 等暂不支持;要求 ClassLiteral Xxx.class
|
||||
if (!(classArg instanceof ClassExpr)) {
|
||||
// 部分 API:parseObject(str, Xxx.class, Feature...) 仍是第 2 参
|
||||
return null;
|
||||
}
|
||||
Type type = ((ClassExpr) classArg).getType();
|
||||
if (!(type instanceof ClassOrInterfaceType)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
ClassOrInterfaceDeclaration enclosing = mce
|
||||
.findAncestor(ClassOrInterfaceDeclaration.class).orElse(null);
|
||||
String enclosingFqn = enclosing == null
|
||||
? "<unknown>"
|
||||
: enclosing.getFullyQualifiedName().orElse(enclosing.getNameAsString());
|
||||
SourceIndex.IndexedType context = index.get(enclosingFqn);
|
||||
|
||||
String fqn = index.resolveFqn(((ClassOrInterfaceType) type).getNameWithScope(), context);
|
||||
if (fqn == null) {
|
||||
fqn = index.resolveFqn(((ClassOrInterfaceType) type).getNameAsString(), context);
|
||||
}
|
||||
if (fqn == null) {
|
||||
return null;
|
||||
}
|
||||
|
||||
CacheReadHint hint = new CacheReadHint();
|
||||
hint.setFilePath(filePath);
|
||||
hint.setLineNumber(mce.getBegin().map(p -> p.line).orElse(0));
|
||||
hint.setEnclosingClass(enclosingFqn);
|
||||
hint.setEnclosingMethod(mce.findAncestor(CallableDeclaration.class)
|
||||
.map(CallableDeclaration::getNameAsString).orElse("<unknown>"));
|
||||
hint.setResolvedValueType(fqn);
|
||||
hint.setRootArray(array || "parseArray".equals(name) || "getJsonToList".equals(name));
|
||||
hint.setConfidence(0.7);
|
||||
|
||||
Expression rawExpr = mce.getArgument(0);
|
||||
Optional<RedisGetRef> getRef = findRedisGetForVar(rawExpr, mce);
|
||||
if (getRef.isPresent()) {
|
||||
hint.setKeyExpression(getRef.get().keyExpr.toString());
|
||||
hint.setResolvedKeyPattern(keyResolver.resolve(
|
||||
getRef.get().keyExpr, enclosing, context));
|
||||
hint.setConfidence(0.85);
|
||||
}
|
||||
return hint;
|
||||
}
|
||||
|
||||
/**
|
||||
* 若 parse 的第 1 参是局部变量,追溯其是否来自 redis get(key)。
|
||||
*/
|
||||
private Optional<RedisGetRef> findRedisGetForVar(Expression rawExpr, MethodCallExpr parseCall) {
|
||||
if (!(rawExpr instanceof NameExpr)) {
|
||||
// 直接 parseObject(redis.get(key), Xxx.class)
|
||||
if (rawExpr instanceof MethodCallExpr) {
|
||||
return extractGetKey((MethodCallExpr) rawExpr);
|
||||
}
|
||||
return Optional.empty();
|
||||
}
|
||||
String varName = ((NameExpr) rawExpr).getNameAsString();
|
||||
Optional<CallableDeclaration> callable = parseCall.findAncestor(CallableDeclaration.class);
|
||||
if (!callable.isPresent()) {
|
||||
return Optional.empty();
|
||||
}
|
||||
for (VariableDeclarator var : callable.get().findAll(VariableDeclarator.class)) {
|
||||
if (!var.getNameAsString().equals(varName) || !var.getInitializer().isPresent()) {
|
||||
continue;
|
||||
}
|
||||
Expression init = var.getInitializer().get();
|
||||
if (init instanceof MethodCallExpr) {
|
||||
return extractGetKey((MethodCallExpr) init);
|
||||
}
|
||||
}
|
||||
return Optional.empty();
|
||||
}
|
||||
|
||||
private Optional<RedisGetRef> extractGetKey(MethodCallExpr call) {
|
||||
String name = call.getNameAsString();
|
||||
String scope = call.getScope().map(Expression::toString).orElse("").toLowerCase(Locale.ROOT);
|
||||
boolean redisScope = scope.contains("redis") || scope.contains("opsforvalue")
|
||||
|| scope.contains("boundvalueops");
|
||||
if ("get".equals(name) || "getString".equals(name)) {
|
||||
if (!redisScope && !REDIS_GET.contains(name)) {
|
||||
// getString 也常见于 RedisUtil
|
||||
if (!"getString".equals(name)) {
|
||||
return Optional.empty();
|
||||
}
|
||||
}
|
||||
if (call.getArguments().isEmpty()) {
|
||||
return Optional.empty();
|
||||
}
|
||||
return Optional.of(new RedisGetRef(call.getArgument(0)));
|
||||
}
|
||||
// redisTemplate.opsForValue().get(key)
|
||||
if ("get".equals(name) && scope.contains("opsforvalue") && !call.getArguments().isEmpty()) {
|
||||
return Optional.of(new RedisGetRef(call.getArgument(0)));
|
||||
}
|
||||
return Optional.empty();
|
||||
}
|
||||
|
||||
private static final class RedisGetRef {
|
||||
final Expression keyExpr;
|
||||
|
||||
RedisGetRef(Expression keyExpr) {
|
||||
this.keyExpr = keyExpr;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -44,7 +44,9 @@ detection:
|
||||
- W03 # stringRedisTemplate.opsForValue().set(key, JsonUtil.getObjectToString(x), ...)
|
||||
- W04 # redisTemplate.opsForValue().set(key, obj, ...)
|
||||
- W05 # redisTemplate.opsForHash().put(key, field, obj)
|
||||
# 类型推断最低置信度,低于此值降级为 P2 提示
|
||||
# W06 读侧辅助:用 parseObject / getJsonToBean 等补强写入点 value 类型(非写入模式)
|
||||
read_hints_enabled: true
|
||||
# 类型推断最低置信度,低于此值标记为低置信度提示
|
||||
min_confidence: 0.6
|
||||
# 字段展开最大深度(防止循环引用)
|
||||
max_field_depth: 8
|
||||
|
||||
71
src/test/java/com/codechecker/cache/detector/CacheReadHintDetectorTest.java
vendored
Normal file
71
src/test/java/com/codechecker/cache/detector/CacheReadHintDetectorTest.java
vendored
Normal file
@@ -0,0 +1,71 @@
|
||||
package com.codechecker.cache.detector;
|
||||
|
||||
import com.codechecker.cache.schema.SourceIndex;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertFalse;
|
||||
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||
|
||||
class CacheReadHintDetectorTest {
|
||||
|
||||
@Test
|
||||
void detectsParseObjectWithRedisGetPairing() {
|
||||
String vo = ""
|
||||
+ "package demo.model;\n"
|
||||
+ "public class FooVo { private String name; }\n";
|
||||
String source = ""
|
||||
+ "package demo;\n"
|
||||
+ "import demo.model.FooVo;\n"
|
||||
+ "public class CacheService {\n"
|
||||
+ " private RedisUtil redisUtil;\n"
|
||||
+ " private static final String KEY = \"demo:foo:\";\n"
|
||||
+ " public FooVo load(String id) {\n"
|
||||
+ " String raw = redisUtil.getString(KEY + id);\n"
|
||||
+ " return JSON.parseObject(raw, FooVo.class);\n"
|
||||
+ " }\n"
|
||||
+ " public void save(String id, FooVo vo) {\n"
|
||||
+ " redisUtil.insert(KEY + id, JSON.toJSONString(vo), 60);\n"
|
||||
+ " }\n"
|
||||
+ "}\n";
|
||||
|
||||
SourceIndex index = new SourceIndex();
|
||||
index.addSource(vo);
|
||||
index.addSource(source);
|
||||
|
||||
List<CacheReadHint> hints = new CacheReadHintDetector(index)
|
||||
.detect("CacheService.java", source);
|
||||
assertEquals(1, hints.size());
|
||||
CacheReadHint hint = hints.get(0);
|
||||
assertEquals("demo.model.FooVo", hint.getResolvedValueType());
|
||||
assertFalse(hint.isRootArray());
|
||||
assertEquals("demo:foo:*", hint.getResolvedKeyPattern());
|
||||
}
|
||||
|
||||
@Test
|
||||
void detectsParseArray() {
|
||||
String vo = ""
|
||||
+ "package demo.model;\n"
|
||||
+ "public class ItemVo { private int rank; }\n";
|
||||
String source = ""
|
||||
+ "package demo;\n"
|
||||
+ "import demo.model.ItemVo;\n"
|
||||
+ "public class ListCache {\n"
|
||||
+ " public List<ItemVo> load(String raw) {\n"
|
||||
+ " return JSON.parseArray(raw, ItemVo.class);\n"
|
||||
+ " }\n"
|
||||
+ "}\n";
|
||||
|
||||
SourceIndex index = new SourceIndex();
|
||||
index.addSource(vo);
|
||||
index.addSource(source);
|
||||
|
||||
List<CacheReadHint> hints = new CacheReadHintDetector(index)
|
||||
.detect("ListCache.java", source);
|
||||
assertEquals(1, hints.size());
|
||||
assertTrue(hints.get(0).isRootArray());
|
||||
assertEquals("demo.model.ItemVo", hints.get(0).getResolvedValueType());
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user