feat: V1.1 - 问题一修复

This commit is contained in:
2026-08-03 11:03:22 +08:00
parent 182e49c4e1
commit 69d581f034
11 changed files with 671 additions and 145 deletions

5
.idea/compiler.xml generated
View File

@@ -1,4 +1,4 @@
<?xml version="1.0" encoding="UTF-8"?>
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="CompilerConfiguration">
<annotationProcessing>
@@ -7,10 +7,11 @@
<sourceTestOutputDir name="target/generated-test-sources/test-annotations" />
<outputRelativeToContentRoot value="true" />
<module name="serialization-schema-checker" />
<module name="redis-schema-checker-parent" />
</profile>
</annotationProcessing>
<bytecodeTargetLevel>
<module name="serialization-schema-checker" target="11" />
</bytecodeTargetLevel>
</component>
</project>
</project>

View File

@@ -0,0 +1,287 @@
# V1.1 非结构写入收敛 — 整改方案
> 版本v1.1
> 日期2026-08-03
> 状态:**已落地**
> 依据:业务仓实跑误报 + 单测复现 `RecordingTodoReplaceScenarioTest`
> 关联:`docs/redis序列化结构检测实施方案.md`、`docs/配置说明.md`
> 约束:**不新增配置项**;收敛为工具默认行为,随 jar 升级生效
---
## 1. 背景与已证实问题
### 1.1 实跑样本jnpf-java-cloud
```text
【序列化结构变更】 jnpf-java-cloud
分支: patrol/dev_patrol_2.0
提交: adbb15c6 → 6ddc72dc
- Key --> recording:todo:replace::
位置: RecordingTodoDomainServiceImpl#replaceText:437
类型: <未解析>
value 新增为: “{}”
- Key --> recording:todo:replace:title::
位置: RecordingTodoDomainServiceImpl#replaceText:452
类型: <未解析>
value 新增为: “{}”
```
业务代码(摘要):
```java
// 缓存替换前原文,纯 String无 JSON / VO 序列化
stringRedisTemplate.opsForValue().set(contentRedisKey, originalContent);
stringRedisTemplate.opsForValue().set(titleRedisKey, originalTitle);
```
### 1.2 复现结论
单测 `RecordingTodoReplaceScenarioTest` 已稳定复现:
| 环节 | 现状 |
|------|------|
| 检测 | 命中 **W04**(直写对象) |
| 类型 | `String` 不在本仓 `SourceIndex``resolvedValueType = null` → 展示 `<未解析>` |
| Schema | `extract(null)` 空结构 → 骨架 `"{}"` |
| 报告 | 旧提交无该写入点 → `WRITE_POINT_ADDED` →「value 新增为」 |
**核心缺口**:新增写入点(`WRITE_POINT_ADDED`)路径上,**没有对「无结构 / 不可展开」的 value 做收敛**;只要 AST 命中 W04/W05及部分边缘写法就会告警。
### 1.3 本方案范围
| 纳入 | 不纳入(另案) |
|------|----------------|
| Redis 非结构写入误报收敛 | MQ `asyncSend` 漏检(作用域过窄) |
| 同类场景枚举与统一收敛规则 | 新增 Key 企微文案 / 整段染色体验 |
| 检测层 + 分析层双闸门设计 | 业务仓 ignore 配置补丁(仅作临时手段) |
---
## 2. 问题本质
工具目标是监控 **可 JSON/对象 Schema 化的序列化结构变更**
当前实现把「能调用 `set/insert/put`」近似当成「值得监控」,收敛只做了很窄的一层:
```text
现有收敛
├─ 方法黑名单setIfAbsent / increment / delete …
├─ isTrivialValue字面量、UUID、toString/valueOf/randomUUID
└─ ignore.key_patternslock / loginCount / Authorization …
缺失收敛
├─ value 声明类型为 String / 数字 / 布尔 / byte[](变量,非字面量)
├─ value 类型无法在本仓展开null / 空 Schema
├─ 先 JSON 序列化到局部 String 再写入Redis 未 unwrap误成 String-W04
└─ WRITE_POINT_ADDED 不检查「是否有可对比结构」
```
因此:**凡是「新增一段无结构写入」都会变成「类型未解析 + value 新增为 {}」**,与 RecordingTodo 同形。
---
## 3. 同类场景排查(是否会中招)
判定标准能否在默认配置W01~W05 全开)下产出与实跑同形的告警
`WRITE_POINT_ADDED` + `<未解析>` 或空/`{}` 骨架,或明显无业务结构)。
### 3.1 高概率同形(与实跑同类,应优先收敛)
| ID | 场景 | 典型代码 | 命中模式 | 结果形态 | 与实跑关系 |
|----|------|----------|----------|----------|------------|
| S1 | **String 变量直写** | `stringRedisTemplate.opsForValue().set(k, originalContent)` | W04 | `<未解析>` + `{}` | **已证实** |
| S2 | **String 方法返回值直写** | `set(k, todo.getContent())` | W04 | 同 S1返回类型 String 时) | 同形;`isTrivialValue` 不认 `getXxx` |
| S3 | **数字 / 布尔变量直写** | `set(k, ttlSeconds)` / `set(k, enabled)` | W04 | `<未解析>` + `{}` | 同形;标量无字段 |
| S4 | **Hash 写入 String/标量** | `opsForHash().put(k, f, nameStr)` | W05 | 同 S1 | W05 仅挡字面量,不挡变量 |
| S5 | **redisUtil.insert 写 String 变量** | `redisUtil.insert(k, token, ttl)` | W04 | 同 S1 | `insert` + 含 redis 的 scope → 走 W04 分支 |
| S6 | **外部依赖类型直写** | `set(k, externalDto)` 且 DTO 不在本仓 | W04 | `<未解析>` + `{}` | 同形;「有对象语义」但工具无法展开,告警无信息量 |
| S7 | **类型擦除 Object / 原始 Map** | `set(k, (Object) x)` / `set(k, map)` 且无法解析元素 | W04 | 常为 `<未解析>` / 空或弱 Schema | 同形或近似 |
### 3.2 中概率变形(会误报,但表现略不同)
| ID | 场景 | 典型代码 | 命中模式 | 结果形态 | 说明 |
|----|------|----------|----------|----------|------|
| S8 | **局部 JSON 字符串再写入** | `String json = JSON.toJSONString(vo); set(k, json)` | W04 | 类型按 String → `<未解析>`+`{}` | Redis **无** `unwrapJsonStringVar`MQ 已有);应归 W01~W03却被当成无结构 |
| S9 | **显式序列化标量** | `set(k, JSON.toJSONString(someString))` | W02 | 偶发低价值点 | 少见unwrap 后仍是 String应丢弃 |
| S10 | **本仓 VO 但字段全 transient / 全忽略注解** | `set(k, emptyVo)` | W04 | 类型有名,骨架仍可能 `{}` | 有类型名,但「新增 {}」仍噪音 |
| S11 | **删除无结构写入点** | 删掉 S1 类代码 | WRITE_POINT_REMOVED | 「原结构 {}」 | 对称噪音,收敛规则应一并覆盖 |
### 3.3 低概率 / 已部分防护(记录边界)
| ID | 场景 | 现状 | 结论 |
|----|------|------|------|
| S12 | 字面量 `"1"` / `UUID.toString()` | `isTrivialValue` 已忽略 | 一般不中招 |
| S13 | key 命中 `*:lock` / `loginCount:*` 等 | `ignore.key_patterns` | 仅覆盖命名约定,**挡不住** `recording:todo:replace:*` |
| S14 | MQ 直传 String / byte[] | `isBareStringOrBytesPayload` 已忽略 | Redis 应对齐MQ 数字变量仍可能漏(见下) |
| S15 | MQ 直传业务 VO | 正常监控 | 不属本问题 |
| S16 | Redis W01~W03 + 本仓 VO | 正常监控 | 不属本问题 |
### 3.4 MQ 侧对照(同类思想,程度不同)
| 点 | Redis现状 | MQ现状 |
|----|---------------|------------|
| 裸 String / byte[] 变量 | **不忽略** → S1 | **忽略** |
| 数字 / 布尔变量 | **不忽略** → S3 | **不忽略**(仅 trivial 字面量)→ **MQ 也有 S3 风险** |
| 局部 `toJSONString` 再 send/set | Redis **不 unwrap** → S8 | **已 unwrap** |
| 无法展开类型仍建点 | 会建点 → S6 | 会建点(直传对象时) |
| WRITE_POINT_ADDED 空 Schema 闸门 | **无** | **无**(同样会「新增 {}」) |
结论:
- Redis 是重灾区;
- MQ 在 String 上已收敛,但 **Integer/Long/Boolean 变量****空 Schema 仍告警** 与 Redis 同类,应在统一规则里一并处理。
### 3.5 分析层放大效应(与模式无关)
`SchemaCheckAnalyzer` 在「新写入点、旧无配对」时:
```text
fileChanged && oldWritePoint == null
→ 无条件 WRITE_POINT_ADDED
→ extract(type) 即使为空也 render → "{}"
→ 进入企微
```
对比「已有写入点」路径:`differ.diff` 空则 `continue`**不会**为空 Schema 刷屏。
因此同类问题在 **新增代码** 时被放大;存量无结构写入若从未变更,不一定出告警。整改必须同时考虑:
1. **检测层少建点**(源头)
2. **分析层对无结构新增/删除做闸门**(兜底)
---
## 4. 收敛规则设计
### 4.1 原则
1. **只监控有结构的序列化 value**(本仓可展开的业务类型,或 unwrap 后得到的业务类型)。
2. **宁漏勿滥**:无法证明「有结构」则不告警。
3. **检测层为主、分析层为辅**:避免脏 WritePoint 进入报告链路。
4. **Redis / MQ 规则对齐**,避免一边收敛一边漏。
### 4.2 规则表(建议落码)
| 规则 | 适用 | 行为 |
|------|------|------|
| R-A 标量类型忽略 | Redis W04/W05MQ 直传 payload | value/payload 声明类型 ∈ `{String, CharSequence, 原始/包装数字, Boolean, UUID, byte/Byte/byte[]}`**不建点** |
| R-B 本仓可展开 | Redis W04/W05MQ 直传对象 | `resolvedFqn == null``SourceIndex` 无此类 → **不建点** |
| R-C 空 Schema 闸门 | AnalyzerADDED / REMOVED | `extract``schema.isEmpty()`**不产生** WRITE_POINT_ADDED/REMOVED字段级 diff 路径保持:空 diff 已 skip |
| R-D 局部 JSON unwrap | Redis对齐 MQ | `String json = toJSONString(x); set/insert(k, json)` → 按 W01~W03 建点,类型为 `x`;若 `x` 仍为标量 → 再走 R-A |
| R-E 方法返回值 | 与 R-A/R-B 相同 | `set(k, obj.getFoo())` 按方法返回类型判定,不因「非字面量」直接放行 |
| R-F 集合 | W04/W05 / MQ | `List<T>` / `Set<T>`:元素 `T` 满足 R-B 才保留;`List<String>` 丢弃 |
### 4.3 明确保留(不应误杀)
| 写法 | 期望 |
|------|------|
| `set(k, JSON.toJSONString(vo))` / `insert` + 序列化 | W01~W03监控 VO |
| `redisTemplate.set(k, demoVo)` 且 DemoVo 在本仓 | W04监控 |
| `opsForHash().put(k, f, demoVo)` 本仓 VO | W05监控 |
| MQ `syncSend/asyncSend(topic, dto)` 本仓 DTO | 监控 |
| 局部 JSON unwrap 后得到本仓 VO | 监控 |
### 4.4 配置策略
**不新增任何 YAML / 检测开关。** R-A~R-F 均为工具内置默认行为,升级 jar 即生效。
业务侧若需临时止血,仍可沿用既有 `ignore.key_patterns` / `ignore.writer_methods` / `suppressions`,但不作为本问题根治手段。
---
## 5. 落地设计
### 5.1 推荐分层
```text
┌─────────────────────────────────────────┐
│ RedisWritePointDetector / MqWritePointDetector
│ R-A 标量忽略
│ R-D Redis 局部 JSON unwrap
│ R-B 本仓类型门槛(直写对象)
│ R-E/R-F 返回值与集合元素
└──────────────────┬──────────────────────┘
┌─────────────────────────────────────────┐
│ SchemaCheckAnalyzer
│ R-C 空 Schema → 跳过 ADDED/REMOVED
│ (字段级 diff 逻辑不变)
└─────────────────────────────────────────┘
```
### 5.2 涉及文件(实施时)
| 文件 | 变更要点 |
|------|----------|
| `RedisWritePointDetector.java` | R-A / R-B / R-D / R-E / R-F |
| `MqWritePointDetector.java` | R-A 扩展数字/布尔;直传对象可加 R-B与 Redis 对齐) |
| `SchemaCheckAnalyzer.java` | R-C 空 Schema 闸门 |
| (可选)抽取 `BareValueTypes` 工具类 | Redis/MQ 共用标量集合,防漂移 |
| `RecordingTodoReplaceScenarioTest` | 修复后期望 **0 写入点**(或拆「修复前/后」) |
| 新增场景单测 | 覆盖 §3.1 / §3.2 表中 S2~S8、S11 |
| `docs/配置说明.md` | 同步 W04/W05 与空 Schema 行为 |
### 5.3 实施顺序建议
本版本一次性落地 R-A~R-FRedis + Analyzer + MQ 对齐),无分阶段开关、无灰度配置。
---
## 6. 测试与验收
### 6.1 复现用例(已有)
- `RecordingTodoReplaceScenarioTest`:当前断言「会误报」。
- 整改后应改为:检测结果为空,或 Analyzer 不产出 keyChanges。
### 6.2 建议补充用例
| 用例 | 期望(整改后) |
|------|----------------|
| `set(k, getContent())` 返回 String | 0 点 |
| `set(k, longVar)` | 0 点 |
| `opsForHash().put(k, f, strVar)` | 0 点 |
| `redisUtil.insert(k, tokenStr, ttl)` | 0 点 |
| `set(k, externalDto)` 不在 SourceIndex | 0 点 |
| `String json = toJSONString(vo); set(k, json)` | 1 点,类型 = VOW02/W01 |
| `set(k, demoVo)` 本仓 VO | 仍 1 点 W04 |
| 删除 String 直写 | 不产生 WRITE_POINT_REMOVED |
| MQ `send(topic, longVar)` | 0 点Phase 4 |
### 6.3 回归
- `TenantScenarioTest`、既有 W01~W05 / MQ fixture 全绿。
- 业务仓抽样RecordingTodo 类告警消失;真实 VO 包装层变更仍能检出。
---
## 7. 临时手段与风险
| 项 | 说明 |
|----|------|
| 无新配置 | 不增加 `keep_empty_schema_*` 等开关;行为固化在检测/分析逻辑 |
| 业务 ignore | 可对 `recording:todo:replace:*` 加 key 忽略,仅止血,不解决 S1~S7 类问题 |
| 误杀风险 | 依赖 jar 内 DTO 的 W04 将被忽略 → 可用既有 `manual_mappings` 或把类型源码纳入仓 |
| Object 擦除 | 可能漏检真实结构 → 接受宁漏勿滥;或要求业务写明 VO 类型 |
| 与「展示优化」关系 | 即使将来给新增 Key 染色,空 `{}` 仍无意义;**必须先收敛再建点** |
---
## 8. 总结
| 问题 | 结论 |
|------|------|
| RecordingTodo 为何告警? | W04 把 String 原文当成对象写入;新增路径无空结构闸门 |
| 是否只有这一处? | **否**。S2~S7 同形S8 变形S11 对称MQ 在数字变量与空 Schema 上同类 |
| 根因一句话 | **新增写入检测缺少「有结构才告警」的收敛** |
| 整改抓手 | 检测层标量/本仓类型收敛 + Analyzer 空 Schema 闸门 + Redis JSON 局部 unwrap |
---
## 9. 检查表
- [x] 认可 R-A~R-F 与「宁漏勿滥」、**不新增配置**
- [x] 外部 jar DTOS6默认不监控
- [x] 代码落地 + `RecordingTodoReplaceScenarioTest` 期望 0 点
- [x] 同步 `配置说明.md`
- [x] 全量 `mvn test` 通过

View File

@@ -226,8 +226,12 @@ mode: block
| W01 | `redisUtil.insert(key, JSON.toJSONString(x), ttl)` | 已启用 |
| W02 | `redisTemplate.opsForValue().set(key, JSON.toJSONString(x), ...)` | 已启用 |
| W03 | `stringRedisTemplate.opsForValue().set(key, JsonUtil.getObjectToString(x), ...)` | 已启用 |
| W04 | `redisTemplate.opsForValue().set(key, obj, ...)` 直写对象 | 已启用Phase 2 |
| W05 | `redisTemplate.opsForHash().put(key, field, obj)` | 已启用Phase 2 |
| W04 | `redisTemplate.opsForValue().set(key, obj, ...)` 直写对象 | 已启用**V1.1** 忽略 `String`/数字/布尔等标量,且 value 须为本仓库可解析类型 |
| W05 | `redisTemplate.opsForHash().put(key, field, obj)` | 已启用;过滤规则同 W04 |
| W01~W03 补充 | 局部 `String json = toJSONString(x)``set/insert` | **V1.1** unwrap 到 `x` |
| 空 Schema 闸门 | 新增/删除写入点且 Schema 为空 | **V1.1** 分析层直接丢弃不发「value 新增为 {}」 |
以上收敛为工具默认行为,**无额外配置开关**。升级 jar 即生效。
业务仓可通过只声明子集暂时关闭某些模式,例如仅保留 JSON 写入:

View File

@@ -6,7 +6,7 @@
<groupId>com.codechecker</groupId>
<artifactId>serialization-schema-checker</artifactId>
<version>1.0.0</version>
<version>1.1.0</version>
<packaging>jar</packaging>
<name>serialization-schema-checker</name>

View File

@@ -167,13 +167,17 @@ public class SchemaCheckAnalyzer {
skeletonRenderer.render(newSchema, protectedPaths(changes),
SkeletonJsonRenderer.DEFAULT_MAX_LEN));
} else if (fileChanged) {
// R-C空 Schema 的新增写入点无结构可告警,收敛丢弃
TypeSchema newSchema = extractorNew.extract(nw.getResolvedValueType(), nw.isRootArray());
if (newSchema == null || newSchema.isEmpty()) {
continue;
}
SchemaChange c = new SchemaChange(ChangeType.WRITE_POINT_ADDED);
fillFromWritePoint(c, nw);
c.setMessage(nw.isMq()
? "新增 MQ 投递点value 类型: " + displayType(nw)
: "新增缓存写入点value 类型: " + displayType(nw));
allChanges.add(c);
TypeSchema newSchema = extractorNew.extract(nw.getResolvedValueType(), nw.isRootArray());
mergeKeyChange(keyChanges, nw,
Collections.singletonList(c),
"",
@@ -187,13 +191,16 @@ public class SchemaCheckAnalyzer {
for (WritePoint ow : oldWps) {
if (!newSigs.contains(ow.signature())
&& !isWritePointIgnored(ow) && !isWriterIgnored(ow)) {
TypeSchema oldSchema = extractorOld.extract(ow.getResolvedValueType(), ow.isRootArray());
if (oldSchema == null || oldSchema.isEmpty()) {
continue;
}
SchemaChange c = new SchemaChange(ChangeType.WRITE_POINT_REMOVED);
fillFromWritePoint(c, ow);
c.setMessage(ow.isMq()
? "删除 MQ 投递点,原 value 类型: " + displayType(ow)
: "删除缓存写入点,原 value 类型: " + displayType(ow));
allChanges.add(c);
TypeSchema oldSchema = extractorOld.extract(ow.getResolvedValueType(), ow.isRootArray());
mergeKeyChange(keyChanges, ow,
Collections.singletonList(c),
skeletonRenderer.render(oldSchema, protectedPaths(

View File

@@ -0,0 +1,54 @@
package com.codechecker.cache.detector;
import com.github.javaparser.ast.type.ArrayType;
import com.github.javaparser.ast.type.Type;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashSet;
import java.util.Set;
/**
* 无业务 JSON 结构的标量 / 裸字节类型Redis / MQ 共用)。这些类型非序列化,直接跳过。
*/
final class BareValueTypes {
private static final Set<String> BARE = Collections.unmodifiableSet(new HashSet<>(Arrays.asList(
"String", "CharSequence", "char", "Character", "byte", "Byte", "int", "long", "short", "double", "float",
"Integer", "Long", "Short", "Double", "Float", "Number", "BigInteger", "BigDecimal", "AtomicInteger", "AtomicLong",
"boolean", "Boolean", "UUID")));
private BareValueTypes() {
}
static boolean isBareSimpleName(String simple) {
return simple != null && BARE.contains(simple);
}
static boolean isBareType(Type type) {
if (type == null) {
return false;
}
if (type.isPrimitiveType()) {
return true;
}
if (type instanceof ArrayType) {
Type component = ((ArrayType) type).getComponentType();
String simple = component.isClassOrInterfaceType() ? component.asClassOrInterfaceType().getNameAsString() : component.asString();
return "byte".equals(simple) || "Byte".equals(simple);
}
if (type.isClassOrInterfaceType()) {
return isBareSimpleName(type.asClassOrInterfaceType().getNameAsString());
}
String asString = type.asString();
return "byte[]".equals(asString) || isBareSimpleName(asString);
}
static boolean isBareFqn(String fqn) {
if (fqn == null || fqn.isEmpty()) {
return false;
}
int dot = fqn.lastIndexOf('.');
return isBareSimpleName(dot >= 0 ? fqn.substring(dot + 1) : fqn);
}
}

View File

@@ -38,6 +38,7 @@ import java.util.Set;
* <li>M2MQ03 convertAndSend、MQ04 MessageBuilder/`Message&lt;T&gt;`、MQ05 JSON 字符串、
* MQ-K03 ProducerRecord、MQ-K04 JSON 字符串</li>
* </ul>
* V1.1:标量 payload 忽略;直传/Message 解包后须为本仓库可展开业务类型。
*/
public class MqWritePointDetector {
@@ -53,8 +54,8 @@ public class MqWritePointDetector {
"List", "ArrayList", "LinkedList", "Set", "HashSet", "Collection"));
private static final Set<String> ENVELOPE_TYPES = new HashSet<>(Arrays.asList(
"Message", "MessageExt", "ProducerRecord"));
private static final Set<String> IGNORE_BARE_TYPES = new HashSet<>(Arrays.asList(
"String", "byte", "Byte"));
/** JSON 字符串模式:仅标量收敛,不强制本仓类型(与 Redis W01~W03 对齐)。 */
private static final Set<String> JSON_STRING_PATTERNS = new HashSet<>(Arrays.asList("MQ05", "MQ-K04"));
private final SourceIndex index;
private final Set<String> enabledPatterns;
@@ -187,6 +188,9 @@ public class MqWritePointDetector {
if (!enabledPatterns.contains(pattern)) {
return null;
}
if (isBareScalarTypeExpr(resolved.typeExpr, mce)) {
return null;
}
WritePoint wp = new WritePoint();
wp.setFilePath(filePath);
@@ -210,9 +214,51 @@ public class MqWritePointDetector {
} else {
wp.setConfidence(0.4);
}
// 直传对象 / Message 解包须本仓库可展开JSON 字符串模式仅拦标量
if (!JSON_STRING_PATTERNS.contains(pattern) && !hasResolvableBusinessType(inferred)) {
return null;
}
return wp;
}
private boolean hasResolvableBusinessType(InferredType inferred) {
if (inferred == null || inferred.fqn == null || inferred.fqn.isEmpty()) {
return false;
}
if (BareValueTypes.isBareFqn(inferred.fqn)) {
return false;
}
return index.get(inferred.fqn) != null;
}
private boolean isBareScalarTypeExpr(Expression expr, MethodCallExpr contextCall) {
if (expr == null) {
return false;
}
if (expr instanceof NameExpr) {
Type declared = findVariableType(((NameExpr) expr).getNameAsString(), contextCall);
return declared != null && BareValueTypes.isBareType(declared);
}
if (expr instanceof CastExpr) {
return BareValueTypes.isBareType(((CastExpr) expr).getType());
}
if (expr instanceof ObjectCreationExpr) {
return BareValueTypes.isBareType(((ObjectCreationExpr) expr).getType());
}
if (expr instanceof MethodCallExpr) {
MethodCallExpr call = (MethodCallExpr) expr;
Optional<ClassOrInterfaceDeclaration> clazz = contextCall.findAncestor(ClassOrInterfaceDeclaration.class);
if (clazz.isPresent()) {
for (MethodDeclaration md : clazz.get().getMethods()) {
if (md.getNameAsString().equals(call.getNameAsString())) {
return BareValueTypes.isBareType(md.getType());
}
}
}
}
return false;
}
/**
* 解析 payloadMessage/MessageBuilderMQ04、JSON 字符串MQ05/MQ-K04、直传对象。
* 无法解析的 Message/纯 String 返回 null忽略
@@ -371,18 +417,7 @@ public class MqWritePointDetector {
if (typeExpr instanceof StringLiteralExpr) {
return true;
}
if (typeExpr instanceof NameExpr) {
Type declared = findVariableType(((NameExpr) typeExpr).getNameAsString(), contextCall);
if (declared != null) {
String simple = declared.isClassOrInterfaceType()
? declared.asClassOrInterfaceType().getNameAsString()
: declared.asString();
if (IGNORE_BARE_TYPES.contains(simple) || "byte[]".equals(declared.asString())) {
return true;
}
}
}
return false;
return isBareScalarTypeExpr(typeExpr, contextCall);
}
private boolean isTrivialValue(Expression expr) {
@@ -451,8 +486,7 @@ public class MqWritePointDetector {
}
if (expr instanceof MethodCallExpr) {
MethodCallExpr call = (MethodCallExpr) expr;
Optional<ClassOrInterfaceDeclaration> clazz = contextCall
.findAncestor(ClassOrInterfaceDeclaration.class);
Optional<ClassOrInterfaceDeclaration> clazz = contextCall.findAncestor(ClassOrInterfaceDeclaration.class);
if (clazz.isPresent()) {
for (MethodDeclaration md : clazz.get().getMethods()) {
if (md.getNameAsString().equals(call.getNameAsString())) {

View File

@@ -9,7 +9,6 @@ import com.github.javaparser.ast.body.ClassOrInterfaceDeclaration;
import com.github.javaparser.ast.body.FieldDeclaration;
import com.github.javaparser.ast.body.MethodDeclaration;
import com.github.javaparser.ast.body.Parameter;
import com.github.javaparser.ast.body.TypeDeclaration;
import com.github.javaparser.ast.body.VariableDeclarator;
import com.github.javaparser.ast.expr.BinaryExpr;
import com.github.javaparser.ast.expr.BooleanLiteralExpr;
@@ -34,6 +33,8 @@ import java.util.Set;
/**
* 从单个 Java 源文件中检测 Redis value 写入点W01~W05
* <p>
* V1.1 收敛:忽略无结构标量;局部 JSON 字符串 unwrapW04/W05 仅保留本仓库可展开业务类型。
*/
public class RedisWritePointDetector {
@@ -48,6 +49,7 @@ public class RedisWritePointDetector {
"randomUUID", "toString", "valueOf"));
private static final Set<String> COLLECTION_SIMPLE = new HashSet<>(Arrays.asList(
"List", "ArrayList", "LinkedList", "Set", "HashSet", "Collection"));
private static final Set<String> DIRECT_OBJECT_PATTERNS = new HashSet<>(Arrays.asList("W04", "W05"));
private final SourceIndex index;
private final Set<String> enabledPatterns;
@@ -97,12 +99,24 @@ public class RedisWritePointDetector {
Expression valueArg = mce.getArgument(1);
Expression serialized = unwrapSerializer(valueArg);
String serializerName = null;
if (serialized != null) {
serializerName = ((MethodCallExpr) valueArg).getNameAsString();
} else {
serialized = unwrapJsonStringVar(valueArg, mce);
if (serialized != null) {
serializerName = serializerMethodOfLocal(valueArg, mce);
}
}
String pattern;
Expression typeExpr;
if (serialized != null) {
pattern = classifySerialized(method, valueArg);
pattern = classifySerialized(method, serializerName);
typeExpr = serialized;
} else if (enabledPatterns.contains("W04") && !isTrivialValue(valueArg)) {
} else if (enabledPatterns.contains("W04")
&& !isTrivialValue(valueArg)
&& !isBareScalarTypeExpr(valueArg, mce)) {
pattern = "W04";
typeExpr = valueArg;
} else {
@@ -111,6 +125,9 @@ public class RedisWritePointDetector {
if (!enabledPatterns.contains(pattern)) {
return null;
}
if (isBareScalarTypeExpr(typeExpr, mce)) {
return null;
}
return buildWritePoint(mce, filePath, pattern, mce.getArgument(0), valueArg, typeExpr);
}
@@ -126,7 +143,13 @@ public class RedisWritePointDetector {
return null;
}
Expression serialized = unwrapSerializer(valueArg);
if (serialized == null) {
serialized = unwrapJsonStringVar(valueArg, mce);
}
Expression typeExpr = serialized != null ? serialized : valueArg;
if (isBareScalarTypeExpr(typeExpr, mce)) {
return null;
}
return buildWritePoint(mce, filePath, "W05", mce.getArgument(0), valueArg, typeExpr);
}
@@ -152,9 +175,23 @@ public class RedisWritePointDetector {
} else {
wp.setConfidence(0.4);
}
if (DIRECT_OBJECT_PATTERNS.contains(pattern) && !hasResolvableBusinessType(inferred)) {
return null;
}
return wp;
}
private boolean hasResolvableBusinessType(InferredType inferred) {
if (inferred == null || inferred.fqn == null || inferred.fqn.isEmpty()) {
return false;
}
if (BareValueTypes.isBareFqn(inferred.fqn)) {
return false;
}
return index.get(inferred.fqn) != null;
}
private boolean isValueOpsScope(MethodCallExpr mce) {
String scope = mce.getScope().map(Expression::toString).orElse("");
String lower = scope.toLowerCase();
@@ -192,9 +229,36 @@ public class RedisWritePointDetector {
}
}
if (expr instanceof ObjectCreationExpr) {
ObjectCreationExpr oce = (ObjectCreationExpr) expr;
String typeName = oce.getType().getNameAsString();
return "UUID".equals(typeName);
return "UUID".equals(((ObjectCreationExpr) expr).getType().getNameAsString());
}
return false;
}
private boolean isBareScalarTypeExpr(Expression expr, MethodCallExpr contextCall) {
if (expr == null) {
return false;
}
if (expr instanceof NameExpr) {
Type declared = findVariableType(((NameExpr) expr).getNameAsString(), contextCall);
return declared != null && BareValueTypes.isBareType(declared);
}
if (expr instanceof CastExpr) {
return BareValueTypes.isBareType(((CastExpr) expr).getType());
}
if (expr instanceof ObjectCreationExpr) {
return BareValueTypes.isBareType(((ObjectCreationExpr) expr).getType());
}
if (expr instanceof MethodCallExpr) {
MethodCallExpr call = (MethodCallExpr) expr;
Optional<ClassOrInterfaceDeclaration> clazz = contextCall
.findAncestor(ClassOrInterfaceDeclaration.class);
if (clazz.isPresent()) {
for (MethodDeclaration md : clazz.get().getMethods()) {
if (md.getNameAsString().equals(call.getNameAsString())) {
return BareValueTypes.isBareType(md.getType());
}
}
}
}
return false;
}
@@ -209,9 +273,52 @@ public class RedisWritePointDetector {
return null;
}
private String classifySerialized(String method, Expression valueArg) {
String serializer = valueArg instanceof MethodCallExpr
? ((MethodCallExpr) valueArg).getNameAsString() : "";
/** 局部 {@code String json = toJSONString(x)} 再写入:回溯到 {@code x}。 */
private Expression unwrapJsonStringVar(Expression valueArg, MethodCallExpr mce) {
if (!(valueArg instanceof NameExpr)) {
return null;
}
String name = ((NameExpr) valueArg).getNameAsString();
Type declared = findVariableType(name, mce);
if (declared != null) {
String simple = declared.isClassOrInterfaceType()
? declared.asClassOrInterfaceType().getNameAsString()
: declared.asString();
if (!"String".equals(simple)) {
return null;
}
}
return unwrapSerializer(findLocalInitializer(name, mce));
}
private String serializerMethodOfLocal(Expression valueArg, MethodCallExpr mce) {
if (!(valueArg instanceof NameExpr)) {
return "";
}
Expression init = findLocalInitializer(((NameExpr) valueArg).getNameAsString(), mce);
if (init instanceof MethodCallExpr) {
String name = ((MethodCallExpr) init).getNameAsString();
if (SERIALIZE_METHODS.contains(name)) {
return name;
}
}
return "";
}
private Expression findLocalInitializer(String name, MethodCallExpr mce) {
Optional<CallableDeclaration> callable = mce.findAncestor(CallableDeclaration.class);
if (!callable.isPresent()) {
return null;
}
for (VariableDeclarator var : callable.get().findAll(VariableDeclarator.class)) {
if (var.getNameAsString().equals(name) && var.getInitializer().isPresent()) {
return var.getInitializer().get();
}
}
return null;
}
private String classifySerialized(String method, String serializer) {
if ("insert".equals(method)) {
return "W01";
}
@@ -237,12 +344,10 @@ public class RedisWritePointDetector {
return resolveTypeNode(((CastExpr) expr).getType(), context);
}
if (expr instanceof ObjectCreationExpr) {
ClassOrInterfaceType t = ((ObjectCreationExpr) expr).getType();
return resolveTypeNode(t, context);
return resolveTypeNode(((ObjectCreationExpr) expr).getType(), context);
}
if (expr instanceof NameExpr) {
String name = ((NameExpr) expr).getNameAsString();
Type declared = findVariableType(name, contextCall);
Type declared = findVariableType(((NameExpr) expr).getNameAsString(), contextCall);
if (declared != null) {
return resolveTypeNode(declared, context);
}

View File

@@ -2,48 +2,27 @@ package com.codechecker.cache;
import com.codechecker.cache.detector.RedisWritePointDetector;
import com.codechecker.cache.detector.WritePoint;
import com.codechecker.cache.diff.ChangeType;
import com.codechecker.cache.diff.SchemaChange;
import com.codechecker.cache.report.CheckReport;
import com.codechecker.cache.report.KeyStructureChange;
import com.codechecker.cache.report.ReportBuilder;
import com.codechecker.cache.schema.JavaSchemaExtractor;
import com.codechecker.cache.schema.SkeletonJsonRenderer;
import com.codechecker.cache.schema.SourceIndex;
import com.codechecker.cache.schema.TypeSchema;
import org.junit.jupiter.api.Test;
import java.util.Arrays;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
/**
* 复现业务仓实跑误报jnpf-java-cloud / RecordingTodoDomainServiceImpl#replaceText
* <pre>
* - Key --> recording:todo:replace::
* 类型: &lt;未解析&gt;
* value 新增为: “{}”
* - Key --> recording:todo:replace:title::
* 类型: &lt;未解析&gt;
* value 新增为: “{}”
* </pre>
* 根因:{@code stringRedisTemplate.opsForValue().set(key, String变量)} 被 W04 命中,
* String 无法展开 Schema → 空骨架 + WRITE_POINT_ADDED。
* 业务仓 RecordingTodoDomainServiceImpl#replaceTextStringRedisTemplate 缓存原文 String。
* V1.1 收敛后不应再检出写入点(修复前会误报 W04 + 类型未解析 + value 新增为 "{}")。
*/
class RecordingTodoReplaceScenarioTest {
private static final Set<String> PATTERNS = new HashSet<>(Arrays.asList("W01", "W02", "W03", "W04", "W05"));
private static final Set<String> PATTERNS = new HashSet<>(Arrays.asList(
"W01", "W02", "W03", "W04", "W05"));
@Test
void reproducesUnresolvedStringCacheFalsePositiveLikeWeCom() {
void ignoresPlainStringCacheWritesAfterConvergence() {
String constants = TestSupport.fixture("fixtures/recording-todo/RecordingTodoConstants.txt");
String service = TestSupport.fixture("fixtures/recording-todo/RecordingTodoDomainServiceImpl.txt");
@@ -51,89 +30,10 @@ class RecordingTodoReplaceScenarioTest {
index.addSource(constants);
index.addSource(service);
List<WritePoint> wps = new RedisWritePointDetector(index, PATTERNS).detect("RecordingTodoDomainServiceImpl.java", service);
wps = wps.stream().sorted(Comparator.comparing(WritePoint::getLineNumber)).collect(Collectors.toList());
List<WritePoint> wps = new RedisWritePointDetector(index, PATTERNS)
.detect("RecordingTodoDomainServiceImpl.java", service);
assertEquals(2, wps.size(), "应检出 content / title 两处 set");
WritePoint contentWp = wps.get(0);
WritePoint titleWp = wps.get(1);
// --- 检测层:与实跑一致 ---
assertEquals("W04", contentWp.getPattern());
assertEquals("W04", titleWp.getPattern());
assertEquals("recording:todo:replace:*:*", contentWp.getResolvedKeyPattern());
assertEquals("recording:todo:replace:title:*:*", titleWp.getResolvedKeyPattern());
assertNull(contentWp.getResolvedValueType(), "String 不在本仓 SourceIndexFQN 应为 null");
assertNull(titleWp.getResolvedValueType());
assertTrue(contentWp.getEnclosingMethod().contains("replaceText") || "replaceText".equals(contentWp.getEnclosingMethod()));
assertTrue(contentWp.getEnclosingClass().endsWith("RecordingTodoDomainServiceImpl"));
// --- Schema空结构 → "{}" ---
JavaSchemaExtractor extractor = new JavaSchemaExtractor(index, 8);
SkeletonJsonRenderer renderer = new SkeletonJsonRenderer();
TypeSchema contentSchema = extractor.extract(contentWp.getResolvedValueType(), contentWp.isRootArray());
TypeSchema titleSchema = extractor.extract(titleWp.getResolvedValueType(), titleWp.isRootArray());
assertTrue(contentSchema.isEmpty());
assertTrue(titleSchema.isEmpty());
assertEquals("{}", renderer.render(contentSchema));
assertEquals("{}", renderer.render(titleSchema));
// --- 报告层复现企微「value 新增为 / 类型未解析」 ---
CheckReport report = buildAddedWritePointReport(contentWp, titleWp, extractor, renderer);
String md = new ReportBuilder("【序列化结构变更】").toMarkdown(report);
assertTrue(md.contains("recording:todo:replace::") || md.contains("recording:todo:replace:*:*"), "应包含 content key企微转义后可为全角实际:\n" + md);
assertTrue(md.contains("recording:todo:replace:title::") || md.contains("recording:todo:replace:title:*:*"), "应包含 title key实际:\n" + md);
assertTrue(md.contains("<未解析>"), "类型应展示为 <未解析>,实际:\n" + md);
assertTrue(md.contains("value 新增为"), "应走新增写入文案,实际:\n" + md);
assertTrue(md.contains("“{}”") || md.contains("\"{}\"") || md.contains("{}"), "骨架应为空对象 {},实际:\n" + md);
assertTrue(md.contains("RecordingTodoDomainServiceImpl#replaceText"), "位置应指向 replaceText实际:\n" + md);
}
/**
* 模拟 SchemaCheckAnalyzer 对「文件新增写入点」的聚合与展示路径(无 git
*/
private static CheckReport buildAddedWritePointReport(WritePoint contentWp, WritePoint titleWp,
JavaSchemaExtractor extractor,
SkeletonJsonRenderer renderer) {
CheckReport report = new CheckReport();
report.setRepository("jnpf-java-cloud");
report.setBranch("patrol/dev_patrol_2.0");
report.setOldSha("adbb15c6");
report.setNewSha("6ddc72dc");
report.setModifier("xgl");
report.setModifyTime("2026-07-29 17:57:02");
for (WritePoint wp : Arrays.asList(contentWp, titleWp)) {
SchemaChange added = new SchemaChange(ChangeType.WRITE_POINT_ADDED);
added.setKeyPattern(wp.getResolvedKeyPattern());
added.setKeyExpression(wp.getKeyExpression());
added.setWriteLocation(wp.location());
added.setValueType(displayType(wp));
added.setMessage("新增缓存写入点value 类型: " + displayType(wp));
report.getChanges().add(added);
TypeSchema schema = extractor.extract(wp.getResolvedValueType(), wp.isRootArray());
KeyStructureChange kc = new KeyStructureChange();
kc.setKeyPattern(wp.getResolvedKeyPattern());
kc.setKeyExpression(wp.getKeyExpression());
kc.setWriteLocation(wp.location());
kc.setValueType(displayType(wp));
kc.setKeyUnresolved(false);
kc.setOldSkeletonJson("");
kc.setNewSkeletonJson(renderer.render(schema, Collections.emptySet(), SkeletonJsonRenderer.DEFAULT_MAX_LEN));
kc.getFieldDetails().add(added);
report.getKeyChanges().add(kc);
}
return report;
}
private static String displayType(WritePoint wp) {
String fqn = wp.getResolvedValueType();
if (fqn == null) {
return "<未解析>";
}
return fqn.contains(".") ? fqn.substring(fqn.lastIndexOf('.') + 1) : fqn;
assertTrue(wps.isEmpty(),
"纯 String 原文缓存不应再告警,实际: " + wps);
}
}

View File

@@ -147,6 +147,23 @@ class MqWritePointDetectorTest {
assertTrue(wp.getResolvedValueType().endsWith("PatrolNotifyVo"));
}
@Test
void ignoresLongAndStringPayload() {
String source = ""
+ "package demo.mq;\n"
+ "import org.apache.rocketmq.spring.core.RocketMQTemplate;\n"
+ "public class BarePayloadProducer {\n"
+ " private RocketMQTemplate rocketMQTemplate;\n"
+ " public void send(String topic, Long count, String text) {\n"
+ " rocketMQTemplate.syncSend(topic, count);\n"
+ " rocketMQTemplate.asyncSend(topic, text);\n"
+ " }\n"
+ "}\n";
List<WritePoint> wps = new MqWritePointDetector(new SourceIndex(), allRocketPatterns())
.detect("BarePayloadProducer.java", source);
assertTrue(wps.isEmpty(), "标量 MQ payload 应忽略,实际: " + wps);
}
private static Set<String> allRocketPatterns() {
Set<String> patterns = new HashSet<>();
patterns.add("MQ01");

View File

@@ -98,4 +98,121 @@ class RedisWritePointDetectorTest {
assertEquals(1, wps.size());
assertEquals("W01", wps.get(0).getPattern());
}
@Test
void ignoresStringVariableW04() {
String source = ""
+ "package demo;\n"
+ "public class TokenCache {\n"
+ " private RedisTemplate<String, String> redisTemplate;\n"
+ " public void save(String key, String tokenVar) {\n"
+ " redisTemplate.opsForValue().set(key, tokenVar, 60, TimeUnit.SECONDS);\n"
+ " }\n"
+ "}\n";
assertTrue(detect(source, new SourceIndex()).isEmpty());
}
@Test
void ignoresLongVariableAndStringGetter() {
String source = ""
+ "package demo;\n"
+ "public class CounterCache {\n"
+ " private RedisTemplate<String, Object> redisTemplate;\n"
+ " public void save(String key, Long count) {\n"
+ " redisTemplate.opsForValue().set(key, count, 60, TimeUnit.SECONDS);\n"
+ " redisTemplate.opsForValue().set(key, getContent(), 60, TimeUnit.SECONDS);\n"
+ " }\n"
+ " private String getContent() { return \"x\"; }\n"
+ "}\n";
assertTrue(detect(source, new SourceIndex()).isEmpty());
}
@Test
void unwrapsLocalJsonStringToW02() {
String vo = ""
+ "package demo.model;\n"
+ "public class DemoVo { private String name; }\n";
String source = ""
+ "package demo;\n"
+ "import demo.model.DemoVo;\n"
+ "public class DemoService {\n"
+ " private RedisTemplate<String, String> redisTemplate;\n"
+ " public void save(String key, DemoVo vo) {\n"
+ " String json = JSON.toJSONString(vo);\n"
+ " redisTemplate.opsForValue().set(key, json, 60, TimeUnit.SECONDS);\n"
+ " }\n"
+ "}\n";
SourceIndex index = new SourceIndex();
index.addSource(vo);
index.addSource(source);
List<WritePoint> wps = detect(source, index);
assertEquals(1, wps.size());
assertEquals("W02", wps.get(0).getPattern());
assertEquals("demo.model.DemoVo", wps.get(0).getResolvedValueType());
}
@Test
void unwrapsLocalJsonStringInsertToW01() {
String vo = ""
+ "package demo.model;\n"
+ "public class DemoVo { private String name; }\n";
String source = ""
+ "package demo;\n"
+ "import demo.model.DemoVo;\n"
+ "public class DemoService {\n"
+ " private RedisUtil redisUtil;\n"
+ " public void save(String key, DemoVo vo) {\n"
+ " String cache = JsonUtil.getObjectToString(vo);\n"
+ " redisUtil.insert(key, cache, 60);\n"
+ " }\n"
+ "}\n";
SourceIndex index = new SourceIndex();
index.addSource(vo);
index.addSource(source);
List<WritePoint> wps = detect(source, index);
assertEquals(1, wps.size());
assertEquals("W01", wps.get(0).getPattern());
assertEquals("demo.model.DemoVo", wps.get(0).getResolvedValueType());
}
@Test
void ignoresHashPutStringAndInsertStringVar() {
String source = ""
+ "package demo;\n"
+ "public class DemoService {\n"
+ " private RedisTemplate<String, Object> redisTemplate;\n"
+ " private RedisUtil redisUtil;\n"
+ " public void save(String key, String field, String nameStr, String token) {\n"
+ " redisTemplate.opsForHash().put(key, field, nameStr);\n"
+ " redisUtil.insert(key, token, 60);\n"
+ " }\n"
+ "}\n";
assertTrue(detect(source, new SourceIndex()).isEmpty());
}
@Test
void ignoresW04WhenTypeNotInSourceIndex() {
String source = ""
+ "package demo;\n"
+ "import external.ExternalDto;\n"
+ "public class DemoService {\n"
+ " private RedisTemplate<String, Object> redisTemplate;\n"
+ " public void save(String key, ExternalDto dto) {\n"
+ " redisTemplate.opsForValue().set(key, dto, 60, TimeUnit.SECONDS);\n"
+ " }\n"
+ "}\n";
assertTrue(detect(source, new SourceIndex()).isEmpty());
}
private static List<WritePoint> detect(String source, SourceIndex index) {
return new RedisWritePointDetector(index, allPatterns()).detect("DemoService.java", source);
}
private static Set<String> allPatterns() {
return new HashSet<>(Arrays.asList("W01", "W02", "W03", "W04", "W05"));
}
}