feat: V1.1 - 问题一修复
This commit is contained in:
3
.idea/compiler.xml
generated
3
.idea/compiler.xml
generated
@@ -1,4 +1,4 @@
|
|||||||
<?xml version="1.0" encoding="UTF-8"?>
|
<?xml version="1.0" encoding="UTF-8"?>
|
||||||
<project version="4">
|
<project version="4">
|
||||||
<component name="CompilerConfiguration">
|
<component name="CompilerConfiguration">
|
||||||
<annotationProcessing>
|
<annotationProcessing>
|
||||||
@@ -7,6 +7,7 @@
|
|||||||
<sourceTestOutputDir name="target/generated-test-sources/test-annotations" />
|
<sourceTestOutputDir name="target/generated-test-sources/test-annotations" />
|
||||||
<outputRelativeToContentRoot value="true" />
|
<outputRelativeToContentRoot value="true" />
|
||||||
<module name="serialization-schema-checker" />
|
<module name="serialization-schema-checker" />
|
||||||
|
<module name="redis-schema-checker-parent" />
|
||||||
</profile>
|
</profile>
|
||||||
</annotationProcessing>
|
</annotationProcessing>
|
||||||
<bytecodeTargetLevel>
|
<bytecodeTargetLevel>
|
||||||
|
|||||||
287
docs/V1.1-非结构写入收敛整改方案.md
Normal file
287
docs/V1.1-非结构写入收敛整改方案.md
Normal file
@@ -0,0 +1,287 @@
|
|||||||
|
# V1.1 非结构写入收敛 — 整改方案
|
||||||
|
|
||||||
|
> 版本:v1.1
|
||||||
|
> 日期:2026-08-03
|
||||||
|
> 状态:**已落地**
|
||||||
|
> 依据:业务仓实跑误报 + 单测复现 `RecordingTodoReplaceScenarioTest`
|
||||||
|
> 关联:`docs/redis序列化结构检测实施方案.md`、`docs/配置说明.md`
|
||||||
|
> 约束:**不新增配置项**;收敛为工具默认行为,随 jar 升级生效
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 背景与已证实问题
|
||||||
|
|
||||||
|
### 1.1 实跑样本(jnpf-java-cloud)
|
||||||
|
|
||||||
|
```text
|
||||||
|
【序列化结构变更】 jnpf-java-cloud
|
||||||
|
分支: patrol/dev_patrol_2.0
|
||||||
|
提交: adbb15c6 → 6ddc72dc
|
||||||
|
- Key --> recording:todo:replace:*:*
|
||||||
|
位置: RecordingTodoDomainServiceImpl#replaceText:437
|
||||||
|
类型: <未解析>
|
||||||
|
value 新增为: “{}”
|
||||||
|
- Key --> recording:todo:replace:title:*:*
|
||||||
|
位置: RecordingTodoDomainServiceImpl#replaceText:452
|
||||||
|
类型: <未解析>
|
||||||
|
value 新增为: “{}”
|
||||||
|
```
|
||||||
|
|
||||||
|
业务代码(摘要):
|
||||||
|
|
||||||
|
```java
|
||||||
|
// 缓存替换前原文,纯 String,无 JSON / VO 序列化
|
||||||
|
stringRedisTemplate.opsForValue().set(contentRedisKey, originalContent);
|
||||||
|
stringRedisTemplate.opsForValue().set(titleRedisKey, originalTitle);
|
||||||
|
```
|
||||||
|
|
||||||
|
### 1.2 复现结论
|
||||||
|
|
||||||
|
单测 `RecordingTodoReplaceScenarioTest` 已稳定复现:
|
||||||
|
|
||||||
|
| 环节 | 现状 |
|
||||||
|
|------|------|
|
||||||
|
| 检测 | 命中 **W04**(直写对象) |
|
||||||
|
| 类型 | `String` 不在本仓 `SourceIndex` → `resolvedValueType = null` → 展示 `<未解析>` |
|
||||||
|
| Schema | `extract(null)` 空结构 → 骨架 `"{}"` |
|
||||||
|
| 报告 | 旧提交无该写入点 → `WRITE_POINT_ADDED` →「value 新增为」 |
|
||||||
|
|
||||||
|
**核心缺口**:新增写入点(`WRITE_POINT_ADDED`)路径上,**没有对「无结构 / 不可展开」的 value 做收敛**;只要 AST 命中 W04/W05(及部分边缘写法),就会告警。
|
||||||
|
|
||||||
|
### 1.3 本方案范围
|
||||||
|
|
||||||
|
| 纳入 | 不纳入(另案) |
|
||||||
|
|------|----------------|
|
||||||
|
| Redis 非结构写入误报收敛 | MQ `asyncSend` 漏检(作用域过窄) |
|
||||||
|
| 同类场景枚举与统一收敛规则 | 新增 Key 企微文案 / 整段染色体验 |
|
||||||
|
| 检测层 + 分析层双闸门设计 | 业务仓 ignore 配置补丁(仅作临时手段) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. 问题本质
|
||||||
|
|
||||||
|
工具目标是监控 **可 JSON/对象 Schema 化的序列化结构变更**。
|
||||||
|
|
||||||
|
当前实现把「能调用 `set/insert/put`」近似当成「值得监控」,收敛只做了很窄的一层:
|
||||||
|
|
||||||
|
```text
|
||||||
|
现有收敛
|
||||||
|
├─ 方法黑名单:setIfAbsent / increment / delete …
|
||||||
|
├─ isTrivialValue:字面量、UUID、toString/valueOf/randomUUID
|
||||||
|
└─ ignore.key_patterns:lock / loginCount / Authorization …
|
||||||
|
|
||||||
|
缺失收敛
|
||||||
|
├─ value 声明类型为 String / 数字 / 布尔 / byte[](变量,非字面量)
|
||||||
|
├─ value 类型无法在本仓展开(null / 空 Schema)
|
||||||
|
├─ 先 JSON 序列化到局部 String 再写入(Redis 未 unwrap,误成 String-W04)
|
||||||
|
└─ WRITE_POINT_ADDED 不检查「是否有可对比结构」
|
||||||
|
```
|
||||||
|
|
||||||
|
因此:**凡是「新增一段无结构写入」都会变成「类型未解析 + value 新增为 {}」**,与 RecordingTodo 同形。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. 同类场景排查(是否会中招)
|
||||||
|
|
||||||
|
判定标准:能否在默认配置(W01~W05 全开)下产出与实跑同形的告警
|
||||||
|
(`WRITE_POINT_ADDED` + `<未解析>` 或空/`{}` 骨架,或明显无业务结构)。
|
||||||
|
|
||||||
|
### 3.1 高概率同形(与实跑同类,应优先收敛)
|
||||||
|
|
||||||
|
| ID | 场景 | 典型代码 | 命中模式 | 结果形态 | 与实跑关系 |
|
||||||
|
|----|------|----------|----------|----------|------------|
|
||||||
|
| S1 | **String 变量直写** | `stringRedisTemplate.opsForValue().set(k, originalContent)` | W04 | `<未解析>` + `{}` | **已证实** |
|
||||||
|
| S2 | **String 方法返回值直写** | `set(k, todo.getContent())` | W04 | 同 S1(返回类型 String 时) | 同形;`isTrivialValue` 不认 `getXxx` |
|
||||||
|
| S3 | **数字 / 布尔变量直写** | `set(k, ttlSeconds)` / `set(k, enabled)` | W04 | `<未解析>` + `{}` | 同形;标量无字段 |
|
||||||
|
| S4 | **Hash 写入 String/标量** | `opsForHash().put(k, f, nameStr)` | W05 | 同 S1 | W05 仅挡字面量,不挡变量 |
|
||||||
|
| S5 | **redisUtil.insert 写 String 变量** | `redisUtil.insert(k, token, ttl)` | W04 | 同 S1 | `insert` + 含 redis 的 scope → 走 W04 分支 |
|
||||||
|
| S6 | **外部依赖类型直写** | `set(k, externalDto)` 且 DTO 不在本仓 | W04 | `<未解析>` + `{}` | 同形;「有对象语义」但工具无法展开,告警无信息量 |
|
||||||
|
| S7 | **类型擦除 Object / 原始 Map** | `set(k, (Object) x)` / `set(k, map)` 且无法解析元素 | W04 | 常为 `<未解析>` / 空或弱 Schema | 同形或近似 |
|
||||||
|
|
||||||
|
### 3.2 中概率变形(会误报,但表现略不同)
|
||||||
|
|
||||||
|
| ID | 场景 | 典型代码 | 命中模式 | 结果形态 | 说明 |
|
||||||
|
|----|------|----------|----------|----------|------|
|
||||||
|
| S8 | **局部 JSON 字符串再写入** | `String json = JSON.toJSONString(vo); set(k, json)` | W04 | 类型按 String → `<未解析>`+`{}` | Redis **无** `unwrapJsonStringVar`(MQ 已有);应归 W01~W03,却被当成无结构 |
|
||||||
|
| S9 | **显式序列化标量** | `set(k, JSON.toJSONString(someString))` | W02 | 偶发低价值点 | 少见;unwrap 后仍是 String,应丢弃 |
|
||||||
|
| S10 | **本仓 VO 但字段全 transient / 全忽略注解** | `set(k, emptyVo)` | W04 | 类型有名,骨架仍可能 `{}` | 有类型名,但「新增 {}」仍噪音 |
|
||||||
|
| S11 | **删除无结构写入点** | 删掉 S1 类代码 | WRITE_POINT_REMOVED | 「原结构 {}」 | 对称噪音,收敛规则应一并覆盖 |
|
||||||
|
|
||||||
|
### 3.3 低概率 / 已部分防护(记录边界)
|
||||||
|
|
||||||
|
| ID | 场景 | 现状 | 结论 |
|
||||||
|
|----|------|------|------|
|
||||||
|
| S12 | 字面量 `"1"` / `UUID.toString()` | `isTrivialValue` 已忽略 | 一般不中招 |
|
||||||
|
| S13 | key 命中 `*:lock` / `loginCount:*` 等 | `ignore.key_patterns` | 仅覆盖命名约定,**挡不住** `recording:todo:replace:*` |
|
||||||
|
| S14 | MQ 直传 String / byte[] | `isBareStringOrBytesPayload` 已忽略 | Redis 应对齐;MQ 数字变量仍可能漏(见下) |
|
||||||
|
| S15 | MQ 直传业务 VO | 正常监控 | 不属本问题 |
|
||||||
|
| S16 | Redis W01~W03 + 本仓 VO | 正常监控 | 不属本问题 |
|
||||||
|
|
||||||
|
### 3.4 MQ 侧对照(同类思想,程度不同)
|
||||||
|
|
||||||
|
| 点 | Redis(现状) | MQ(现状) |
|
||||||
|
|----|---------------|------------|
|
||||||
|
| 裸 String / byte[] 变量 | **不忽略** → S1 | **忽略** |
|
||||||
|
| 数字 / 布尔变量 | **不忽略** → S3 | **不忽略**(仅 trivial 字面量)→ **MQ 也有 S3 风险** |
|
||||||
|
| 局部 `toJSONString` 再 send/set | Redis **不 unwrap** → S8 | **已 unwrap** |
|
||||||
|
| 无法展开类型仍建点 | 会建点 → S6 | 会建点(直传对象时) |
|
||||||
|
| WRITE_POINT_ADDED 空 Schema 闸门 | **无** | **无**(同样会「新增 {}」) |
|
||||||
|
|
||||||
|
结论:
|
||||||
|
- Redis 是重灾区;
|
||||||
|
- MQ 在 String 上已收敛,但 **Integer/Long/Boolean 变量** 与 **空 Schema 仍告警** 与 Redis 同类,应在统一规则里一并处理。
|
||||||
|
|
||||||
|
### 3.5 分析层放大效应(与模式无关)
|
||||||
|
|
||||||
|
`SchemaCheckAnalyzer` 在「新写入点、旧无配对」时:
|
||||||
|
|
||||||
|
```text
|
||||||
|
fileChanged && oldWritePoint == null
|
||||||
|
→ 无条件 WRITE_POINT_ADDED
|
||||||
|
→ extract(type) 即使为空也 render → "{}"
|
||||||
|
→ 进入企微
|
||||||
|
```
|
||||||
|
|
||||||
|
对比「已有写入点」路径:`differ.diff` 空则 `continue`,**不会**为空 Schema 刷屏。
|
||||||
|
|
||||||
|
因此同类问题在 **新增代码** 时被放大;存量无结构写入若从未变更,不一定出告警。整改必须同时考虑:
|
||||||
|
|
||||||
|
1. **检测层少建点**(源头)
|
||||||
|
2. **分析层对无结构新增/删除做闸门**(兜底)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. 收敛规则设计
|
||||||
|
|
||||||
|
### 4.1 原则
|
||||||
|
|
||||||
|
1. **只监控有结构的序列化 value**(本仓可展开的业务类型,或 unwrap 后得到的业务类型)。
|
||||||
|
2. **宁漏勿滥**:无法证明「有结构」则不告警。
|
||||||
|
3. **检测层为主、分析层为辅**:避免脏 WritePoint 进入报告链路。
|
||||||
|
4. **Redis / MQ 规则对齐**,避免一边收敛一边漏。
|
||||||
|
|
||||||
|
### 4.2 规则表(建议落码)
|
||||||
|
|
||||||
|
| 规则 | 适用 | 行为 |
|
||||||
|
|------|------|------|
|
||||||
|
| R-A 标量类型忽略 | Redis W04/W05;MQ 直传 payload | value/payload 声明类型 ∈ `{String, CharSequence, 原始/包装数字, Boolean, UUID, byte/Byte/byte[]}` → **不建点** |
|
||||||
|
| R-B 本仓可展开 | Redis W04/W05;MQ 直传对象 | `resolvedFqn == null` 或 `SourceIndex` 无此类 → **不建点** |
|
||||||
|
| R-C 空 Schema 闸门 | Analyzer:ADDED / REMOVED | `extract` 后 `schema.isEmpty()` → **不产生** WRITE_POINT_ADDED/REMOVED(字段级 diff 路径保持:空 diff 已 skip) |
|
||||||
|
| R-D 局部 JSON unwrap | Redis(对齐 MQ) | `String json = toJSONString(x); set/insert(k, json)` → 按 W01~W03 建点,类型为 `x`;若 `x` 仍为标量 → 再走 R-A |
|
||||||
|
| R-E 方法返回值 | 与 R-A/R-B 相同 | `set(k, obj.getFoo())` 按方法返回类型判定,不因「非字面量」直接放行 |
|
||||||
|
| R-F 集合 | W04/W05 / MQ | `List<T>` / `Set<T>`:元素 `T` 满足 R-B 才保留;`List<String>` 丢弃 |
|
||||||
|
|
||||||
|
### 4.3 明确保留(不应误杀)
|
||||||
|
|
||||||
|
| 写法 | 期望 |
|
||||||
|
|------|------|
|
||||||
|
| `set(k, JSON.toJSONString(vo))` / `insert` + 序列化 | W01~W03,监控 VO |
|
||||||
|
| `redisTemplate.set(k, demoVo)` 且 DemoVo 在本仓 | W04,监控 |
|
||||||
|
| `opsForHash().put(k, f, demoVo)` 本仓 VO | W05,监控 |
|
||||||
|
| MQ `syncSend/asyncSend(topic, dto)` 本仓 DTO | 监控 |
|
||||||
|
| 局部 JSON unwrap 后得到本仓 VO | 监控 |
|
||||||
|
|
||||||
|
### 4.4 配置策略
|
||||||
|
|
||||||
|
**不新增任何 YAML / 检测开关。** R-A~R-F 均为工具内置默认行为,升级 jar 即生效。
|
||||||
|
|
||||||
|
业务侧若需临时止血,仍可沿用既有 `ignore.key_patterns` / `ignore.writer_methods` / `suppressions`,但不作为本问题根治手段。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. 落地设计
|
||||||
|
|
||||||
|
### 5.1 推荐分层
|
||||||
|
|
||||||
|
```text
|
||||||
|
┌─────────────────────────────────────────┐
|
||||||
|
│ RedisWritePointDetector / MqWritePointDetector
|
||||||
|
│ R-A 标量忽略
|
||||||
|
│ R-D Redis 局部 JSON unwrap
|
||||||
|
│ R-B 本仓类型门槛(直写对象)
|
||||||
|
│ R-E/R-F 返回值与集合元素
|
||||||
|
└──────────────────┬──────────────────────┘
|
||||||
|
▼
|
||||||
|
┌─────────────────────────────────────────┐
|
||||||
|
│ SchemaCheckAnalyzer
|
||||||
|
│ R-C 空 Schema → 跳过 ADDED/REMOVED
|
||||||
|
│ (字段级 diff 逻辑不变)
|
||||||
|
└─────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.2 涉及文件(实施时)
|
||||||
|
|
||||||
|
| 文件 | 变更要点 |
|
||||||
|
|------|----------|
|
||||||
|
| `RedisWritePointDetector.java` | R-A / R-B / R-D / R-E / R-F |
|
||||||
|
| `MqWritePointDetector.java` | R-A 扩展数字/布尔;直传对象可加 R-B(与 Redis 对齐) |
|
||||||
|
| `SchemaCheckAnalyzer.java` | R-C 空 Schema 闸门 |
|
||||||
|
| (可选)抽取 `BareValueTypes` 工具类 | Redis/MQ 共用标量集合,防漂移 |
|
||||||
|
| `RecordingTodoReplaceScenarioTest` | 修复后期望 **0 写入点**(或拆「修复前/后」) |
|
||||||
|
| 新增场景单测 | 覆盖 §3.1 / §3.2 表中 S2~S8、S11 |
|
||||||
|
| `docs/配置说明.md` | 同步 W04/W05 与空 Schema 行为 |
|
||||||
|
|
||||||
|
### 5.3 实施顺序建议
|
||||||
|
|
||||||
|
本版本一次性落地 R-A~R-F(Redis + Analyzer + MQ 对齐),无分阶段开关、无灰度配置。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. 测试与验收
|
||||||
|
|
||||||
|
### 6.1 复现用例(已有)
|
||||||
|
|
||||||
|
- `RecordingTodoReplaceScenarioTest`:当前断言「会误报」。
|
||||||
|
- 整改后应改为:检测结果为空,或 Analyzer 不产出 keyChanges。
|
||||||
|
|
||||||
|
### 6.2 建议补充用例
|
||||||
|
|
||||||
|
| 用例 | 期望(整改后) |
|
||||||
|
|------|----------------|
|
||||||
|
| `set(k, getContent())` 返回 String | 0 点 |
|
||||||
|
| `set(k, longVar)` | 0 点 |
|
||||||
|
| `opsForHash().put(k, f, strVar)` | 0 点 |
|
||||||
|
| `redisUtil.insert(k, tokenStr, ttl)` | 0 点 |
|
||||||
|
| `set(k, externalDto)` 不在 SourceIndex | 0 点 |
|
||||||
|
| `String json = toJSONString(vo); set(k, json)` | 1 点,类型 = VO,W02/W01 |
|
||||||
|
| `set(k, demoVo)` 本仓 VO | 仍 1 点 W04 |
|
||||||
|
| 删除 String 直写 | 不产生 WRITE_POINT_REMOVED |
|
||||||
|
| MQ `send(topic, longVar)` | 0 点(Phase 4) |
|
||||||
|
|
||||||
|
### 6.3 回归
|
||||||
|
|
||||||
|
- `TenantScenarioTest`、既有 W01~W05 / MQ fixture 全绿。
|
||||||
|
- 业务仓抽样:RecordingTodo 类告警消失;真实 VO 包装层变更仍能检出。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. 临时手段与风险
|
||||||
|
|
||||||
|
| 项 | 说明 |
|
||||||
|
|----|------|
|
||||||
|
| 无新配置 | 不增加 `keep_empty_schema_*` 等开关;行为固化在检测/分析逻辑 |
|
||||||
|
| 业务 ignore | 可对 `recording:todo:replace:*` 加 key 忽略,仅止血,不解决 S1~S7 类问题 |
|
||||||
|
| 误杀风险 | 依赖 jar 内 DTO 的 W04 将被忽略 → 可用既有 `manual_mappings` 或把类型源码纳入仓 |
|
||||||
|
| Object 擦除 | 可能漏检真实结构 → 接受宁漏勿滥;或要求业务写明 VO 类型 |
|
||||||
|
| 与「展示优化」关系 | 即使将来给新增 Key 染色,空 `{}` 仍无意义;**必须先收敛再建点** |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. 总结
|
||||||
|
|
||||||
|
| 问题 | 结论 |
|
||||||
|
|------|------|
|
||||||
|
| RecordingTodo 为何告警? | W04 把 String 原文当成对象写入;新增路径无空结构闸门 |
|
||||||
|
| 是否只有这一处? | **否**。S2~S7 同形,S8 变形,S11 对称;MQ 在数字变量与空 Schema 上同类 |
|
||||||
|
| 根因一句话 | **新增写入检测缺少「有结构才告警」的收敛** |
|
||||||
|
| 整改抓手 | 检测层标量/本仓类型收敛 + Analyzer 空 Schema 闸门 + Redis JSON 局部 unwrap |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. 检查表
|
||||||
|
|
||||||
|
- [x] 认可 R-A~R-F 与「宁漏勿滥」、**不新增配置**
|
||||||
|
- [x] 外部 jar DTO(S6)默认不监控
|
||||||
|
- [x] 代码落地 + `RecordingTodoReplaceScenarioTest` 期望 0 点
|
||||||
|
- [x] 同步 `配置说明.md`
|
||||||
|
- [x] 全量 `mvn test` 通过
|
||||||
@@ -226,8 +226,12 @@ mode: block
|
|||||||
| W01 | `redisUtil.insert(key, JSON.toJSONString(x), ttl)` | 已启用 |
|
| W01 | `redisUtil.insert(key, JSON.toJSONString(x), ttl)` | 已启用 |
|
||||||
| W02 | `redisTemplate.opsForValue().set(key, JSON.toJSONString(x), ...)` | 已启用 |
|
| W02 | `redisTemplate.opsForValue().set(key, JSON.toJSONString(x), ...)` | 已启用 |
|
||||||
| W03 | `stringRedisTemplate.opsForValue().set(key, JsonUtil.getObjectToString(x), ...)` | 已启用 |
|
| W03 | `stringRedisTemplate.opsForValue().set(key, JsonUtil.getObjectToString(x), ...)` | 已启用 |
|
||||||
| W04 | `redisTemplate.opsForValue().set(key, obj, ...)` 直写对象 | 已启用(Phase 2) |
|
| W04 | `redisTemplate.opsForValue().set(key, obj, ...)` 直写对象 | 已启用;**V1.1** 忽略 `String`/数字/布尔等标量,且 value 须为本仓库可解析类型 |
|
||||||
| W05 | `redisTemplate.opsForHash().put(key, field, obj)` | 已启用(Phase 2) |
|
| W05 | `redisTemplate.opsForHash().put(key, field, obj)` | 已启用;过滤规则同 W04 |
|
||||||
|
| W01~W03 补充 | 局部 `String json = toJSONString(x)` 再 `set/insert` | **V1.1** unwrap 到 `x` |
|
||||||
|
| 空 Schema 闸门 | 新增/删除写入点且 Schema 为空 | **V1.1** 分析层直接丢弃,不发「value 新增为 {}」 |
|
||||||
|
|
||||||
|
以上收敛为工具默认行为,**无额外配置开关**。升级 jar 即生效。
|
||||||
|
|
||||||
业务仓可通过只声明子集暂时关闭某些模式,例如仅保留 JSON 写入:
|
业务仓可通过只声明子集暂时关闭某些模式,例如仅保留 JSON 写入:
|
||||||
|
|
||||||
|
|||||||
2
pom.xml
2
pom.xml
@@ -6,7 +6,7 @@
|
|||||||
|
|
||||||
<groupId>com.codechecker</groupId>
|
<groupId>com.codechecker</groupId>
|
||||||
<artifactId>serialization-schema-checker</artifactId>
|
<artifactId>serialization-schema-checker</artifactId>
|
||||||
<version>1.0.0</version>
|
<version>1.1.0</version>
|
||||||
<packaging>jar</packaging>
|
<packaging>jar</packaging>
|
||||||
|
|
||||||
<name>serialization-schema-checker</name>
|
<name>serialization-schema-checker</name>
|
||||||
|
|||||||
@@ -167,13 +167,17 @@ public class SchemaCheckAnalyzer {
|
|||||||
skeletonRenderer.render(newSchema, protectedPaths(changes),
|
skeletonRenderer.render(newSchema, protectedPaths(changes),
|
||||||
SkeletonJsonRenderer.DEFAULT_MAX_LEN));
|
SkeletonJsonRenderer.DEFAULT_MAX_LEN));
|
||||||
} else if (fileChanged) {
|
} else if (fileChanged) {
|
||||||
|
// R-C:空 Schema 的新增写入点无结构可告警,收敛丢弃
|
||||||
|
TypeSchema newSchema = extractorNew.extract(nw.getResolvedValueType(), nw.isRootArray());
|
||||||
|
if (newSchema == null || newSchema.isEmpty()) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
SchemaChange c = new SchemaChange(ChangeType.WRITE_POINT_ADDED);
|
SchemaChange c = new SchemaChange(ChangeType.WRITE_POINT_ADDED);
|
||||||
fillFromWritePoint(c, nw);
|
fillFromWritePoint(c, nw);
|
||||||
c.setMessage(nw.isMq()
|
c.setMessage(nw.isMq()
|
||||||
? "新增 MQ 投递点,value 类型: " + displayType(nw)
|
? "新增 MQ 投递点,value 类型: " + displayType(nw)
|
||||||
: "新增缓存写入点,value 类型: " + displayType(nw));
|
: "新增缓存写入点,value 类型: " + displayType(nw));
|
||||||
allChanges.add(c);
|
allChanges.add(c);
|
||||||
TypeSchema newSchema = extractorNew.extract(nw.getResolvedValueType(), nw.isRootArray());
|
|
||||||
mergeKeyChange(keyChanges, nw,
|
mergeKeyChange(keyChanges, nw,
|
||||||
Collections.singletonList(c),
|
Collections.singletonList(c),
|
||||||
"",
|
"",
|
||||||
@@ -187,13 +191,16 @@ public class SchemaCheckAnalyzer {
|
|||||||
for (WritePoint ow : oldWps) {
|
for (WritePoint ow : oldWps) {
|
||||||
if (!newSigs.contains(ow.signature())
|
if (!newSigs.contains(ow.signature())
|
||||||
&& !isWritePointIgnored(ow) && !isWriterIgnored(ow)) {
|
&& !isWritePointIgnored(ow) && !isWriterIgnored(ow)) {
|
||||||
|
TypeSchema oldSchema = extractorOld.extract(ow.getResolvedValueType(), ow.isRootArray());
|
||||||
|
if (oldSchema == null || oldSchema.isEmpty()) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
SchemaChange c = new SchemaChange(ChangeType.WRITE_POINT_REMOVED);
|
SchemaChange c = new SchemaChange(ChangeType.WRITE_POINT_REMOVED);
|
||||||
fillFromWritePoint(c, ow);
|
fillFromWritePoint(c, ow);
|
||||||
c.setMessage(ow.isMq()
|
c.setMessage(ow.isMq()
|
||||||
? "删除 MQ 投递点,原 value 类型: " + displayType(ow)
|
? "删除 MQ 投递点,原 value 类型: " + displayType(ow)
|
||||||
: "删除缓存写入点,原 value 类型: " + displayType(ow));
|
: "删除缓存写入点,原 value 类型: " + displayType(ow));
|
||||||
allChanges.add(c);
|
allChanges.add(c);
|
||||||
TypeSchema oldSchema = extractorOld.extract(ow.getResolvedValueType(), ow.isRootArray());
|
|
||||||
mergeKeyChange(keyChanges, ow,
|
mergeKeyChange(keyChanges, ow,
|
||||||
Collections.singletonList(c),
|
Collections.singletonList(c),
|
||||||
skeletonRenderer.render(oldSchema, protectedPaths(
|
skeletonRenderer.render(oldSchema, protectedPaths(
|
||||||
|
|||||||
54
src/main/java/com/codechecker/cache/detector/BareValueTypes.java
vendored
Normal file
54
src/main/java/com/codechecker/cache/detector/BareValueTypes.java
vendored
Normal file
@@ -0,0 +1,54 @@
|
|||||||
|
package com.codechecker.cache.detector;
|
||||||
|
|
||||||
|
import com.github.javaparser.ast.type.ArrayType;
|
||||||
|
import com.github.javaparser.ast.type.Type;
|
||||||
|
|
||||||
|
import java.util.Arrays;
|
||||||
|
import java.util.Collections;
|
||||||
|
import java.util.HashSet;
|
||||||
|
import java.util.Set;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 无业务 JSON 结构的标量 / 裸字节类型(Redis / MQ 共用)。这些类型非序列化,直接跳过。
|
||||||
|
*/
|
||||||
|
final class BareValueTypes {
|
||||||
|
|
||||||
|
private static final Set<String> BARE = Collections.unmodifiableSet(new HashSet<>(Arrays.asList(
|
||||||
|
"String", "CharSequence", "char", "Character", "byte", "Byte", "int", "long", "short", "double", "float",
|
||||||
|
"Integer", "Long", "Short", "Double", "Float", "Number", "BigInteger", "BigDecimal", "AtomicInteger", "AtomicLong",
|
||||||
|
"boolean", "Boolean", "UUID")));
|
||||||
|
|
||||||
|
private BareValueTypes() {
|
||||||
|
}
|
||||||
|
|
||||||
|
static boolean isBareSimpleName(String simple) {
|
||||||
|
return simple != null && BARE.contains(simple);
|
||||||
|
}
|
||||||
|
|
||||||
|
static boolean isBareType(Type type) {
|
||||||
|
if (type == null) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
if (type.isPrimitiveType()) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (type instanceof ArrayType) {
|
||||||
|
Type component = ((ArrayType) type).getComponentType();
|
||||||
|
String simple = component.isClassOrInterfaceType() ? component.asClassOrInterfaceType().getNameAsString() : component.asString();
|
||||||
|
return "byte".equals(simple) || "Byte".equals(simple);
|
||||||
|
}
|
||||||
|
if (type.isClassOrInterfaceType()) {
|
||||||
|
return isBareSimpleName(type.asClassOrInterfaceType().getNameAsString());
|
||||||
|
}
|
||||||
|
String asString = type.asString();
|
||||||
|
return "byte[]".equals(asString) || isBareSimpleName(asString);
|
||||||
|
}
|
||||||
|
|
||||||
|
static boolean isBareFqn(String fqn) {
|
||||||
|
if (fqn == null || fqn.isEmpty()) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
int dot = fqn.lastIndexOf('.');
|
||||||
|
return isBareSimpleName(dot >= 0 ? fqn.substring(dot + 1) : fqn);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -38,6 +38,7 @@ import java.util.Set;
|
|||||||
* <li>M2:MQ03 convertAndSend、MQ04 MessageBuilder/`Message<T>`、MQ05 JSON 字符串、
|
* <li>M2:MQ03 convertAndSend、MQ04 MessageBuilder/`Message<T>`、MQ05 JSON 字符串、
|
||||||
* MQ-K03 ProducerRecord、MQ-K04 JSON 字符串</li>
|
* MQ-K03 ProducerRecord、MQ-K04 JSON 字符串</li>
|
||||||
* </ul>
|
* </ul>
|
||||||
|
* V1.1:标量 payload 忽略;直传/Message 解包后须为本仓库可展开业务类型。
|
||||||
*/
|
*/
|
||||||
public class MqWritePointDetector {
|
public class MqWritePointDetector {
|
||||||
|
|
||||||
@@ -53,8 +54,8 @@ public class MqWritePointDetector {
|
|||||||
"List", "ArrayList", "LinkedList", "Set", "HashSet", "Collection"));
|
"List", "ArrayList", "LinkedList", "Set", "HashSet", "Collection"));
|
||||||
private static final Set<String> ENVELOPE_TYPES = new HashSet<>(Arrays.asList(
|
private static final Set<String> ENVELOPE_TYPES = new HashSet<>(Arrays.asList(
|
||||||
"Message", "MessageExt", "ProducerRecord"));
|
"Message", "MessageExt", "ProducerRecord"));
|
||||||
private static final Set<String> IGNORE_BARE_TYPES = new HashSet<>(Arrays.asList(
|
/** JSON 字符串模式:仅标量收敛,不强制本仓类型(与 Redis W01~W03 对齐)。 */
|
||||||
"String", "byte", "Byte"));
|
private static final Set<String> JSON_STRING_PATTERNS = new HashSet<>(Arrays.asList("MQ05", "MQ-K04"));
|
||||||
|
|
||||||
private final SourceIndex index;
|
private final SourceIndex index;
|
||||||
private final Set<String> enabledPatterns;
|
private final Set<String> enabledPatterns;
|
||||||
@@ -187,6 +188,9 @@ public class MqWritePointDetector {
|
|||||||
if (!enabledPatterns.contains(pattern)) {
|
if (!enabledPatterns.contains(pattern)) {
|
||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
|
if (isBareScalarTypeExpr(resolved.typeExpr, mce)) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
WritePoint wp = new WritePoint();
|
WritePoint wp = new WritePoint();
|
||||||
wp.setFilePath(filePath);
|
wp.setFilePath(filePath);
|
||||||
@@ -210,9 +214,51 @@ public class MqWritePointDetector {
|
|||||||
} else {
|
} else {
|
||||||
wp.setConfidence(0.4);
|
wp.setConfidence(0.4);
|
||||||
}
|
}
|
||||||
|
// 直传对象 / Message 解包:须本仓库可展开;JSON 字符串模式仅拦标量
|
||||||
|
if (!JSON_STRING_PATTERNS.contains(pattern) && !hasResolvableBusinessType(inferred)) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
return wp;
|
return wp;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private boolean hasResolvableBusinessType(InferredType inferred) {
|
||||||
|
if (inferred == null || inferred.fqn == null || inferred.fqn.isEmpty()) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
if (BareValueTypes.isBareFqn(inferred.fqn)) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
return index.get(inferred.fqn) != null;
|
||||||
|
}
|
||||||
|
|
||||||
|
private boolean isBareScalarTypeExpr(Expression expr, MethodCallExpr contextCall) {
|
||||||
|
if (expr == null) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
if (expr instanceof NameExpr) {
|
||||||
|
Type declared = findVariableType(((NameExpr) expr).getNameAsString(), contextCall);
|
||||||
|
return declared != null && BareValueTypes.isBareType(declared);
|
||||||
|
}
|
||||||
|
if (expr instanceof CastExpr) {
|
||||||
|
return BareValueTypes.isBareType(((CastExpr) expr).getType());
|
||||||
|
}
|
||||||
|
if (expr instanceof ObjectCreationExpr) {
|
||||||
|
return BareValueTypes.isBareType(((ObjectCreationExpr) expr).getType());
|
||||||
|
}
|
||||||
|
if (expr instanceof MethodCallExpr) {
|
||||||
|
MethodCallExpr call = (MethodCallExpr) expr;
|
||||||
|
Optional<ClassOrInterfaceDeclaration> clazz = contextCall.findAncestor(ClassOrInterfaceDeclaration.class);
|
||||||
|
if (clazz.isPresent()) {
|
||||||
|
for (MethodDeclaration md : clazz.get().getMethods()) {
|
||||||
|
if (md.getNameAsString().equals(call.getNameAsString())) {
|
||||||
|
return BareValueTypes.isBareType(md.getType());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 解析 payload:Message/MessageBuilder(MQ04)、JSON 字符串(MQ05/MQ-K04)、直传对象。
|
* 解析 payload:Message/MessageBuilder(MQ04)、JSON 字符串(MQ05/MQ-K04)、直传对象。
|
||||||
* 无法解析的 Message/纯 String 返回 null(忽略)。
|
* 无法解析的 Message/纯 String 返回 null(忽略)。
|
||||||
@@ -371,18 +417,7 @@ public class MqWritePointDetector {
|
|||||||
if (typeExpr instanceof StringLiteralExpr) {
|
if (typeExpr instanceof StringLiteralExpr) {
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
if (typeExpr instanceof NameExpr) {
|
return isBareScalarTypeExpr(typeExpr, contextCall);
|
||||||
Type declared = findVariableType(((NameExpr) typeExpr).getNameAsString(), contextCall);
|
|
||||||
if (declared != null) {
|
|
||||||
String simple = declared.isClassOrInterfaceType()
|
|
||||||
? declared.asClassOrInterfaceType().getNameAsString()
|
|
||||||
: declared.asString();
|
|
||||||
if (IGNORE_BARE_TYPES.contains(simple) || "byte[]".equals(declared.asString())) {
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return false;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
private boolean isTrivialValue(Expression expr) {
|
private boolean isTrivialValue(Expression expr) {
|
||||||
@@ -451,8 +486,7 @@ public class MqWritePointDetector {
|
|||||||
}
|
}
|
||||||
if (expr instanceof MethodCallExpr) {
|
if (expr instanceof MethodCallExpr) {
|
||||||
MethodCallExpr call = (MethodCallExpr) expr;
|
MethodCallExpr call = (MethodCallExpr) expr;
|
||||||
Optional<ClassOrInterfaceDeclaration> clazz = contextCall
|
Optional<ClassOrInterfaceDeclaration> clazz = contextCall.findAncestor(ClassOrInterfaceDeclaration.class);
|
||||||
.findAncestor(ClassOrInterfaceDeclaration.class);
|
|
||||||
if (clazz.isPresent()) {
|
if (clazz.isPresent()) {
|
||||||
for (MethodDeclaration md : clazz.get().getMethods()) {
|
for (MethodDeclaration md : clazz.get().getMethods()) {
|
||||||
if (md.getNameAsString().equals(call.getNameAsString())) {
|
if (md.getNameAsString().equals(call.getNameAsString())) {
|
||||||
|
|||||||
@@ -9,7 +9,6 @@ import com.github.javaparser.ast.body.ClassOrInterfaceDeclaration;
|
|||||||
import com.github.javaparser.ast.body.FieldDeclaration;
|
import com.github.javaparser.ast.body.FieldDeclaration;
|
||||||
import com.github.javaparser.ast.body.MethodDeclaration;
|
import com.github.javaparser.ast.body.MethodDeclaration;
|
||||||
import com.github.javaparser.ast.body.Parameter;
|
import com.github.javaparser.ast.body.Parameter;
|
||||||
import com.github.javaparser.ast.body.TypeDeclaration;
|
|
||||||
import com.github.javaparser.ast.body.VariableDeclarator;
|
import com.github.javaparser.ast.body.VariableDeclarator;
|
||||||
import com.github.javaparser.ast.expr.BinaryExpr;
|
import com.github.javaparser.ast.expr.BinaryExpr;
|
||||||
import com.github.javaparser.ast.expr.BooleanLiteralExpr;
|
import com.github.javaparser.ast.expr.BooleanLiteralExpr;
|
||||||
@@ -34,6 +33,8 @@ import java.util.Set;
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* 从单个 Java 源文件中检测 Redis value 写入点(W01~W05)。
|
* 从单个 Java 源文件中检测 Redis value 写入点(W01~W05)。
|
||||||
|
* <p>
|
||||||
|
* V1.1 收敛:忽略无结构标量;局部 JSON 字符串 unwrap;W04/W05 仅保留本仓库可展开业务类型。
|
||||||
*/
|
*/
|
||||||
public class RedisWritePointDetector {
|
public class RedisWritePointDetector {
|
||||||
|
|
||||||
@@ -48,6 +49,7 @@ public class RedisWritePointDetector {
|
|||||||
"randomUUID", "toString", "valueOf"));
|
"randomUUID", "toString", "valueOf"));
|
||||||
private static final Set<String> COLLECTION_SIMPLE = new HashSet<>(Arrays.asList(
|
private static final Set<String> COLLECTION_SIMPLE = new HashSet<>(Arrays.asList(
|
||||||
"List", "ArrayList", "LinkedList", "Set", "HashSet", "Collection"));
|
"List", "ArrayList", "LinkedList", "Set", "HashSet", "Collection"));
|
||||||
|
private static final Set<String> DIRECT_OBJECT_PATTERNS = new HashSet<>(Arrays.asList("W04", "W05"));
|
||||||
|
|
||||||
private final SourceIndex index;
|
private final SourceIndex index;
|
||||||
private final Set<String> enabledPatterns;
|
private final Set<String> enabledPatterns;
|
||||||
@@ -97,12 +99,24 @@ public class RedisWritePointDetector {
|
|||||||
|
|
||||||
Expression valueArg = mce.getArgument(1);
|
Expression valueArg = mce.getArgument(1);
|
||||||
Expression serialized = unwrapSerializer(valueArg);
|
Expression serialized = unwrapSerializer(valueArg);
|
||||||
|
String serializerName = null;
|
||||||
|
if (serialized != null) {
|
||||||
|
serializerName = ((MethodCallExpr) valueArg).getNameAsString();
|
||||||
|
} else {
|
||||||
|
serialized = unwrapJsonStringVar(valueArg, mce);
|
||||||
|
if (serialized != null) {
|
||||||
|
serializerName = serializerMethodOfLocal(valueArg, mce);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
String pattern;
|
String pattern;
|
||||||
Expression typeExpr;
|
Expression typeExpr;
|
||||||
if (serialized != null) {
|
if (serialized != null) {
|
||||||
pattern = classifySerialized(method, valueArg);
|
pattern = classifySerialized(method, serializerName);
|
||||||
typeExpr = serialized;
|
typeExpr = serialized;
|
||||||
} else if (enabledPatterns.contains("W04") && !isTrivialValue(valueArg)) {
|
} else if (enabledPatterns.contains("W04")
|
||||||
|
&& !isTrivialValue(valueArg)
|
||||||
|
&& !isBareScalarTypeExpr(valueArg, mce)) {
|
||||||
pattern = "W04";
|
pattern = "W04";
|
||||||
typeExpr = valueArg;
|
typeExpr = valueArg;
|
||||||
} else {
|
} else {
|
||||||
@@ -111,6 +125,9 @@ public class RedisWritePointDetector {
|
|||||||
if (!enabledPatterns.contains(pattern)) {
|
if (!enabledPatterns.contains(pattern)) {
|
||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
|
if (isBareScalarTypeExpr(typeExpr, mce)) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
return buildWritePoint(mce, filePath, pattern, mce.getArgument(0), valueArg, typeExpr);
|
return buildWritePoint(mce, filePath, pattern, mce.getArgument(0), valueArg, typeExpr);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -126,7 +143,13 @@ public class RedisWritePointDetector {
|
|||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
Expression serialized = unwrapSerializer(valueArg);
|
Expression serialized = unwrapSerializer(valueArg);
|
||||||
|
if (serialized == null) {
|
||||||
|
serialized = unwrapJsonStringVar(valueArg, mce);
|
||||||
|
}
|
||||||
Expression typeExpr = serialized != null ? serialized : valueArg;
|
Expression typeExpr = serialized != null ? serialized : valueArg;
|
||||||
|
if (isBareScalarTypeExpr(typeExpr, mce)) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
return buildWritePoint(mce, filePath, "W05", mce.getArgument(0), valueArg, typeExpr);
|
return buildWritePoint(mce, filePath, "W05", mce.getArgument(0), valueArg, typeExpr);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -152,9 +175,23 @@ public class RedisWritePointDetector {
|
|||||||
} else {
|
} else {
|
||||||
wp.setConfidence(0.4);
|
wp.setConfidence(0.4);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (DIRECT_OBJECT_PATTERNS.contains(pattern) && !hasResolvableBusinessType(inferred)) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
return wp;
|
return wp;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private boolean hasResolvableBusinessType(InferredType inferred) {
|
||||||
|
if (inferred == null || inferred.fqn == null || inferred.fqn.isEmpty()) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
if (BareValueTypes.isBareFqn(inferred.fqn)) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
return index.get(inferred.fqn) != null;
|
||||||
|
}
|
||||||
|
|
||||||
private boolean isValueOpsScope(MethodCallExpr mce) {
|
private boolean isValueOpsScope(MethodCallExpr mce) {
|
||||||
String scope = mce.getScope().map(Expression::toString).orElse("");
|
String scope = mce.getScope().map(Expression::toString).orElse("");
|
||||||
String lower = scope.toLowerCase();
|
String lower = scope.toLowerCase();
|
||||||
@@ -192,9 +229,36 @@ public class RedisWritePointDetector {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
if (expr instanceof ObjectCreationExpr) {
|
if (expr instanceof ObjectCreationExpr) {
|
||||||
ObjectCreationExpr oce = (ObjectCreationExpr) expr;
|
return "UUID".equals(((ObjectCreationExpr) expr).getType().getNameAsString());
|
||||||
String typeName = oce.getType().getNameAsString();
|
}
|
||||||
return "UUID".equals(typeName);
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
private boolean isBareScalarTypeExpr(Expression expr, MethodCallExpr contextCall) {
|
||||||
|
if (expr == null) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
if (expr instanceof NameExpr) {
|
||||||
|
Type declared = findVariableType(((NameExpr) expr).getNameAsString(), contextCall);
|
||||||
|
return declared != null && BareValueTypes.isBareType(declared);
|
||||||
|
}
|
||||||
|
if (expr instanceof CastExpr) {
|
||||||
|
return BareValueTypes.isBareType(((CastExpr) expr).getType());
|
||||||
|
}
|
||||||
|
if (expr instanceof ObjectCreationExpr) {
|
||||||
|
return BareValueTypes.isBareType(((ObjectCreationExpr) expr).getType());
|
||||||
|
}
|
||||||
|
if (expr instanceof MethodCallExpr) {
|
||||||
|
MethodCallExpr call = (MethodCallExpr) expr;
|
||||||
|
Optional<ClassOrInterfaceDeclaration> clazz = contextCall
|
||||||
|
.findAncestor(ClassOrInterfaceDeclaration.class);
|
||||||
|
if (clazz.isPresent()) {
|
||||||
|
for (MethodDeclaration md : clazz.get().getMethods()) {
|
||||||
|
if (md.getNameAsString().equals(call.getNameAsString())) {
|
||||||
|
return BareValueTypes.isBareType(md.getType());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
@@ -209,9 +273,52 @@ public class RedisWritePointDetector {
|
|||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
|
|
||||||
private String classifySerialized(String method, Expression valueArg) {
|
/** 局部 {@code String json = toJSONString(x)} 再写入:回溯到 {@code x}。 */
|
||||||
String serializer = valueArg instanceof MethodCallExpr
|
private Expression unwrapJsonStringVar(Expression valueArg, MethodCallExpr mce) {
|
||||||
? ((MethodCallExpr) valueArg).getNameAsString() : "";
|
if (!(valueArg instanceof NameExpr)) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
String name = ((NameExpr) valueArg).getNameAsString();
|
||||||
|
Type declared = findVariableType(name, mce);
|
||||||
|
if (declared != null) {
|
||||||
|
String simple = declared.isClassOrInterfaceType()
|
||||||
|
? declared.asClassOrInterfaceType().getNameAsString()
|
||||||
|
: declared.asString();
|
||||||
|
if (!"String".equals(simple)) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return unwrapSerializer(findLocalInitializer(name, mce));
|
||||||
|
}
|
||||||
|
|
||||||
|
private String serializerMethodOfLocal(Expression valueArg, MethodCallExpr mce) {
|
||||||
|
if (!(valueArg instanceof NameExpr)) {
|
||||||
|
return "";
|
||||||
|
}
|
||||||
|
Expression init = findLocalInitializer(((NameExpr) valueArg).getNameAsString(), mce);
|
||||||
|
if (init instanceof MethodCallExpr) {
|
||||||
|
String name = ((MethodCallExpr) init).getNameAsString();
|
||||||
|
if (SERIALIZE_METHODS.contains(name)) {
|
||||||
|
return name;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "";
|
||||||
|
}
|
||||||
|
|
||||||
|
private Expression findLocalInitializer(String name, MethodCallExpr mce) {
|
||||||
|
Optional<CallableDeclaration> callable = mce.findAncestor(CallableDeclaration.class);
|
||||||
|
if (!callable.isPresent()) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
for (VariableDeclarator var : callable.get().findAll(VariableDeclarator.class)) {
|
||||||
|
if (var.getNameAsString().equals(name) && var.getInitializer().isPresent()) {
|
||||||
|
return var.getInitializer().get();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
private String classifySerialized(String method, String serializer) {
|
||||||
if ("insert".equals(method)) {
|
if ("insert".equals(method)) {
|
||||||
return "W01";
|
return "W01";
|
||||||
}
|
}
|
||||||
@@ -237,12 +344,10 @@ public class RedisWritePointDetector {
|
|||||||
return resolveTypeNode(((CastExpr) expr).getType(), context);
|
return resolveTypeNode(((CastExpr) expr).getType(), context);
|
||||||
}
|
}
|
||||||
if (expr instanceof ObjectCreationExpr) {
|
if (expr instanceof ObjectCreationExpr) {
|
||||||
ClassOrInterfaceType t = ((ObjectCreationExpr) expr).getType();
|
return resolveTypeNode(((ObjectCreationExpr) expr).getType(), context);
|
||||||
return resolveTypeNode(t, context);
|
|
||||||
}
|
}
|
||||||
if (expr instanceof NameExpr) {
|
if (expr instanceof NameExpr) {
|
||||||
String name = ((NameExpr) expr).getNameAsString();
|
Type declared = findVariableType(((NameExpr) expr).getNameAsString(), contextCall);
|
||||||
Type declared = findVariableType(name, contextCall);
|
|
||||||
if (declared != null) {
|
if (declared != null) {
|
||||||
return resolveTypeNode(declared, context);
|
return resolveTypeNode(declared, context);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -2,48 +2,27 @@ package com.codechecker.cache;
|
|||||||
|
|
||||||
import com.codechecker.cache.detector.RedisWritePointDetector;
|
import com.codechecker.cache.detector.RedisWritePointDetector;
|
||||||
import com.codechecker.cache.detector.WritePoint;
|
import com.codechecker.cache.detector.WritePoint;
|
||||||
import com.codechecker.cache.diff.ChangeType;
|
|
||||||
import com.codechecker.cache.diff.SchemaChange;
|
|
||||||
import com.codechecker.cache.report.CheckReport;
|
|
||||||
import com.codechecker.cache.report.KeyStructureChange;
|
|
||||||
import com.codechecker.cache.report.ReportBuilder;
|
|
||||||
import com.codechecker.cache.schema.JavaSchemaExtractor;
|
|
||||||
import com.codechecker.cache.schema.SkeletonJsonRenderer;
|
|
||||||
import com.codechecker.cache.schema.SourceIndex;
|
import com.codechecker.cache.schema.SourceIndex;
|
||||||
import com.codechecker.cache.schema.TypeSchema;
|
|
||||||
import org.junit.jupiter.api.Test;
|
import org.junit.jupiter.api.Test;
|
||||||
|
|
||||||
import java.util.Arrays;
|
import java.util.Arrays;
|
||||||
import java.util.Collections;
|
|
||||||
import java.util.Comparator;
|
|
||||||
import java.util.HashSet;
|
import java.util.HashSet;
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
import java.util.Set;
|
import java.util.Set;
|
||||||
import java.util.stream.Collectors;
|
|
||||||
|
|
||||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
|
||||||
import static org.junit.jupiter.api.Assertions.assertNull;
|
|
||||||
import static org.junit.jupiter.api.Assertions.assertTrue;
|
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 复现业务仓实跑误报(jnpf-java-cloud / RecordingTodoDomainServiceImpl#replaceText):
|
* 业务仓 RecordingTodoDomainServiceImpl#replaceText:StringRedisTemplate 缓存原文 String。
|
||||||
* <pre>
|
* V1.1 收敛后不应再检出写入点(修复前会误报 W04 + 类型未解析 + value 新增为 "{}")。
|
||||||
* - Key --> recording:todo:replace:*:*
|
|
||||||
* 类型: <未解析>
|
|
||||||
* value 新增为: “{}”
|
|
||||||
* - Key --> recording:todo:replace:title:*:*
|
|
||||||
* 类型: <未解析>
|
|
||||||
* value 新增为: “{}”
|
|
||||||
* </pre>
|
|
||||||
* 根因:{@code stringRedisTemplate.opsForValue().set(key, String变量)} 被 W04 命中,
|
|
||||||
* String 无法展开 Schema → 空骨架 + WRITE_POINT_ADDED。
|
|
||||||
*/
|
*/
|
||||||
class RecordingTodoReplaceScenarioTest {
|
class RecordingTodoReplaceScenarioTest {
|
||||||
|
|
||||||
private static final Set<String> PATTERNS = new HashSet<>(Arrays.asList("W01", "W02", "W03", "W04", "W05"));
|
private static final Set<String> PATTERNS = new HashSet<>(Arrays.asList(
|
||||||
|
"W01", "W02", "W03", "W04", "W05"));
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
void reproducesUnresolvedStringCacheFalsePositiveLikeWeCom() {
|
void ignoresPlainStringCacheWritesAfterConvergence() {
|
||||||
String constants = TestSupport.fixture("fixtures/recording-todo/RecordingTodoConstants.txt");
|
String constants = TestSupport.fixture("fixtures/recording-todo/RecordingTodoConstants.txt");
|
||||||
String service = TestSupport.fixture("fixtures/recording-todo/RecordingTodoDomainServiceImpl.txt");
|
String service = TestSupport.fixture("fixtures/recording-todo/RecordingTodoDomainServiceImpl.txt");
|
||||||
|
|
||||||
@@ -51,89 +30,10 @@ class RecordingTodoReplaceScenarioTest {
|
|||||||
index.addSource(constants);
|
index.addSource(constants);
|
||||||
index.addSource(service);
|
index.addSource(service);
|
||||||
|
|
||||||
List<WritePoint> wps = new RedisWritePointDetector(index, PATTERNS).detect("RecordingTodoDomainServiceImpl.java", service);
|
List<WritePoint> wps = new RedisWritePointDetector(index, PATTERNS)
|
||||||
wps = wps.stream().sorted(Comparator.comparing(WritePoint::getLineNumber)).collect(Collectors.toList());
|
.detect("RecordingTodoDomainServiceImpl.java", service);
|
||||||
|
|
||||||
assertEquals(2, wps.size(), "应检出 content / title 两处 set");
|
assertTrue(wps.isEmpty(),
|
||||||
|
"纯 String 原文缓存不应再告警,实际: " + wps);
|
||||||
WritePoint contentWp = wps.get(0);
|
|
||||||
WritePoint titleWp = wps.get(1);
|
|
||||||
|
|
||||||
// --- 检测层:与实跑一致 ---
|
|
||||||
assertEquals("W04", contentWp.getPattern());
|
|
||||||
assertEquals("W04", titleWp.getPattern());
|
|
||||||
assertEquals("recording:todo:replace:*:*", contentWp.getResolvedKeyPattern());
|
|
||||||
assertEquals("recording:todo:replace:title:*:*", titleWp.getResolvedKeyPattern());
|
|
||||||
assertNull(contentWp.getResolvedValueType(), "String 不在本仓 SourceIndex,FQN 应为 null");
|
|
||||||
assertNull(titleWp.getResolvedValueType());
|
|
||||||
assertTrue(contentWp.getEnclosingMethod().contains("replaceText") || "replaceText".equals(contentWp.getEnclosingMethod()));
|
|
||||||
assertTrue(contentWp.getEnclosingClass().endsWith("RecordingTodoDomainServiceImpl"));
|
|
||||||
|
|
||||||
// --- Schema:空结构 → "{}" ---
|
|
||||||
JavaSchemaExtractor extractor = new JavaSchemaExtractor(index, 8);
|
|
||||||
SkeletonJsonRenderer renderer = new SkeletonJsonRenderer();
|
|
||||||
TypeSchema contentSchema = extractor.extract(contentWp.getResolvedValueType(), contentWp.isRootArray());
|
|
||||||
TypeSchema titleSchema = extractor.extract(titleWp.getResolvedValueType(), titleWp.isRootArray());
|
|
||||||
assertTrue(contentSchema.isEmpty());
|
|
||||||
assertTrue(titleSchema.isEmpty());
|
|
||||||
assertEquals("{}", renderer.render(contentSchema));
|
|
||||||
assertEquals("{}", renderer.render(titleSchema));
|
|
||||||
|
|
||||||
// --- 报告层:复现企微「value 新增为 / 类型未解析」 ---
|
|
||||||
CheckReport report = buildAddedWritePointReport(contentWp, titleWp, extractor, renderer);
|
|
||||||
String md = new ReportBuilder("【序列化结构变更】").toMarkdown(report);
|
|
||||||
|
|
||||||
assertTrue(md.contains("recording:todo:replace:*:*") || md.contains("recording:todo:replace:*:*"), "应包含 content key(企微转义后可为全角*),实际:\n" + md);
|
|
||||||
assertTrue(md.contains("recording:todo:replace:title:*:*") || md.contains("recording:todo:replace:title:*:*"), "应包含 title key,实际:\n" + md);
|
|
||||||
assertTrue(md.contains("<未解析>"), "类型应展示为 <未解析>,实际:\n" + md);
|
|
||||||
assertTrue(md.contains("value 新增为"), "应走新增写入文案,实际:\n" + md);
|
|
||||||
assertTrue(md.contains("“{}”") || md.contains("\"{}\"") || md.contains("{}"), "骨架应为空对象 {},实际:\n" + md);
|
|
||||||
assertTrue(md.contains("RecordingTodoDomainServiceImpl#replaceText"), "位置应指向 replaceText,实际:\n" + md);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 模拟 SchemaCheckAnalyzer 对「文件新增写入点」的聚合与展示路径(无 git)。
|
|
||||||
*/
|
|
||||||
private static CheckReport buildAddedWritePointReport(WritePoint contentWp, WritePoint titleWp,
|
|
||||||
JavaSchemaExtractor extractor,
|
|
||||||
SkeletonJsonRenderer renderer) {
|
|
||||||
CheckReport report = new CheckReport();
|
|
||||||
report.setRepository("jnpf-java-cloud");
|
|
||||||
report.setBranch("patrol/dev_patrol_2.0");
|
|
||||||
report.setOldSha("adbb15c6");
|
|
||||||
report.setNewSha("6ddc72dc");
|
|
||||||
report.setModifier("xgl");
|
|
||||||
report.setModifyTime("2026-07-29 17:57:02");
|
|
||||||
|
|
||||||
for (WritePoint wp : Arrays.asList(contentWp, titleWp)) {
|
|
||||||
SchemaChange added = new SchemaChange(ChangeType.WRITE_POINT_ADDED);
|
|
||||||
added.setKeyPattern(wp.getResolvedKeyPattern());
|
|
||||||
added.setKeyExpression(wp.getKeyExpression());
|
|
||||||
added.setWriteLocation(wp.location());
|
|
||||||
added.setValueType(displayType(wp));
|
|
||||||
added.setMessage("新增缓存写入点,value 类型: " + displayType(wp));
|
|
||||||
report.getChanges().add(added);
|
|
||||||
|
|
||||||
TypeSchema schema = extractor.extract(wp.getResolvedValueType(), wp.isRootArray());
|
|
||||||
KeyStructureChange kc = new KeyStructureChange();
|
|
||||||
kc.setKeyPattern(wp.getResolvedKeyPattern());
|
|
||||||
kc.setKeyExpression(wp.getKeyExpression());
|
|
||||||
kc.setWriteLocation(wp.location());
|
|
||||||
kc.setValueType(displayType(wp));
|
|
||||||
kc.setKeyUnresolved(false);
|
|
||||||
kc.setOldSkeletonJson("");
|
|
||||||
kc.setNewSkeletonJson(renderer.render(schema, Collections.emptySet(), SkeletonJsonRenderer.DEFAULT_MAX_LEN));
|
|
||||||
kc.getFieldDetails().add(added);
|
|
||||||
report.getKeyChanges().add(kc);
|
|
||||||
}
|
|
||||||
return report;
|
|
||||||
}
|
|
||||||
|
|
||||||
private static String displayType(WritePoint wp) {
|
|
||||||
String fqn = wp.getResolvedValueType();
|
|
||||||
if (fqn == null) {
|
|
||||||
return "<未解析>";
|
|
||||||
}
|
|
||||||
return fqn.contains(".") ? fqn.substring(fqn.lastIndexOf('.') + 1) : fqn;
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -147,6 +147,23 @@ class MqWritePointDetectorTest {
|
|||||||
assertTrue(wp.getResolvedValueType().endsWith("PatrolNotifyVo"));
|
assertTrue(wp.getResolvedValueType().endsWith("PatrolNotifyVo"));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void ignoresLongAndStringPayload() {
|
||||||
|
String source = ""
|
||||||
|
+ "package demo.mq;\n"
|
||||||
|
+ "import org.apache.rocketmq.spring.core.RocketMQTemplate;\n"
|
||||||
|
+ "public class BarePayloadProducer {\n"
|
||||||
|
+ " private RocketMQTemplate rocketMQTemplate;\n"
|
||||||
|
+ " public void send(String topic, Long count, String text) {\n"
|
||||||
|
+ " rocketMQTemplate.syncSend(topic, count);\n"
|
||||||
|
+ " rocketMQTemplate.asyncSend(topic, text);\n"
|
||||||
|
+ " }\n"
|
||||||
|
+ "}\n";
|
||||||
|
List<WritePoint> wps = new MqWritePointDetector(new SourceIndex(), allRocketPatterns())
|
||||||
|
.detect("BarePayloadProducer.java", source);
|
||||||
|
assertTrue(wps.isEmpty(), "标量 MQ payload 应忽略,实际: " + wps);
|
||||||
|
}
|
||||||
|
|
||||||
private static Set<String> allRocketPatterns() {
|
private static Set<String> allRocketPatterns() {
|
||||||
Set<String> patterns = new HashSet<>();
|
Set<String> patterns = new HashSet<>();
|
||||||
patterns.add("MQ01");
|
patterns.add("MQ01");
|
||||||
|
|||||||
@@ -98,4 +98,121 @@ class RedisWritePointDetectorTest {
|
|||||||
assertEquals(1, wps.size());
|
assertEquals(1, wps.size());
|
||||||
assertEquals("W01", wps.get(0).getPattern());
|
assertEquals("W01", wps.get(0).getPattern());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void ignoresStringVariableW04() {
|
||||||
|
String source = ""
|
||||||
|
+ "package demo;\n"
|
||||||
|
+ "public class TokenCache {\n"
|
||||||
|
+ " private RedisTemplate<String, String> redisTemplate;\n"
|
||||||
|
+ " public void save(String key, String tokenVar) {\n"
|
||||||
|
+ " redisTemplate.opsForValue().set(key, tokenVar, 60, TimeUnit.SECONDS);\n"
|
||||||
|
+ " }\n"
|
||||||
|
+ "}\n";
|
||||||
|
assertTrue(detect(source, new SourceIndex()).isEmpty());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void ignoresLongVariableAndStringGetter() {
|
||||||
|
String source = ""
|
||||||
|
+ "package demo;\n"
|
||||||
|
+ "public class CounterCache {\n"
|
||||||
|
+ " private RedisTemplate<String, Object> redisTemplate;\n"
|
||||||
|
+ " public void save(String key, Long count) {\n"
|
||||||
|
+ " redisTemplate.opsForValue().set(key, count, 60, TimeUnit.SECONDS);\n"
|
||||||
|
+ " redisTemplate.opsForValue().set(key, getContent(), 60, TimeUnit.SECONDS);\n"
|
||||||
|
+ " }\n"
|
||||||
|
+ " private String getContent() { return \"x\"; }\n"
|
||||||
|
+ "}\n";
|
||||||
|
assertTrue(detect(source, new SourceIndex()).isEmpty());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void unwrapsLocalJsonStringToW02() {
|
||||||
|
String vo = ""
|
||||||
|
+ "package demo.model;\n"
|
||||||
|
+ "public class DemoVo { private String name; }\n";
|
||||||
|
String source = ""
|
||||||
|
+ "package demo;\n"
|
||||||
|
+ "import demo.model.DemoVo;\n"
|
||||||
|
+ "public class DemoService {\n"
|
||||||
|
+ " private RedisTemplate<String, String> redisTemplate;\n"
|
||||||
|
+ " public void save(String key, DemoVo vo) {\n"
|
||||||
|
+ " String json = JSON.toJSONString(vo);\n"
|
||||||
|
+ " redisTemplate.opsForValue().set(key, json, 60, TimeUnit.SECONDS);\n"
|
||||||
|
+ " }\n"
|
||||||
|
+ "}\n";
|
||||||
|
|
||||||
|
SourceIndex index = new SourceIndex();
|
||||||
|
index.addSource(vo);
|
||||||
|
index.addSource(source);
|
||||||
|
|
||||||
|
List<WritePoint> wps = detect(source, index);
|
||||||
|
assertEquals(1, wps.size());
|
||||||
|
assertEquals("W02", wps.get(0).getPattern());
|
||||||
|
assertEquals("demo.model.DemoVo", wps.get(0).getResolvedValueType());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void unwrapsLocalJsonStringInsertToW01() {
|
||||||
|
String vo = ""
|
||||||
|
+ "package demo.model;\n"
|
||||||
|
+ "public class DemoVo { private String name; }\n";
|
||||||
|
String source = ""
|
||||||
|
+ "package demo;\n"
|
||||||
|
+ "import demo.model.DemoVo;\n"
|
||||||
|
+ "public class DemoService {\n"
|
||||||
|
+ " private RedisUtil redisUtil;\n"
|
||||||
|
+ " public void save(String key, DemoVo vo) {\n"
|
||||||
|
+ " String cache = JsonUtil.getObjectToString(vo);\n"
|
||||||
|
+ " redisUtil.insert(key, cache, 60);\n"
|
||||||
|
+ " }\n"
|
||||||
|
+ "}\n";
|
||||||
|
|
||||||
|
SourceIndex index = new SourceIndex();
|
||||||
|
index.addSource(vo);
|
||||||
|
index.addSource(source);
|
||||||
|
|
||||||
|
List<WritePoint> wps = detect(source, index);
|
||||||
|
assertEquals(1, wps.size());
|
||||||
|
assertEquals("W01", wps.get(0).getPattern());
|
||||||
|
assertEquals("demo.model.DemoVo", wps.get(0).getResolvedValueType());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void ignoresHashPutStringAndInsertStringVar() {
|
||||||
|
String source = ""
|
||||||
|
+ "package demo;\n"
|
||||||
|
+ "public class DemoService {\n"
|
||||||
|
+ " private RedisTemplate<String, Object> redisTemplate;\n"
|
||||||
|
+ " private RedisUtil redisUtil;\n"
|
||||||
|
+ " public void save(String key, String field, String nameStr, String token) {\n"
|
||||||
|
+ " redisTemplate.opsForHash().put(key, field, nameStr);\n"
|
||||||
|
+ " redisUtil.insert(key, token, 60);\n"
|
||||||
|
+ " }\n"
|
||||||
|
+ "}\n";
|
||||||
|
assertTrue(detect(source, new SourceIndex()).isEmpty());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void ignoresW04WhenTypeNotInSourceIndex() {
|
||||||
|
String source = ""
|
||||||
|
+ "package demo;\n"
|
||||||
|
+ "import external.ExternalDto;\n"
|
||||||
|
+ "public class DemoService {\n"
|
||||||
|
+ " private RedisTemplate<String, Object> redisTemplate;\n"
|
||||||
|
+ " public void save(String key, ExternalDto dto) {\n"
|
||||||
|
+ " redisTemplate.opsForValue().set(key, dto, 60, TimeUnit.SECONDS);\n"
|
||||||
|
+ " }\n"
|
||||||
|
+ "}\n";
|
||||||
|
assertTrue(detect(source, new SourceIndex()).isEmpty());
|
||||||
|
}
|
||||||
|
|
||||||
|
private static List<WritePoint> detect(String source, SourceIndex index) {
|
||||||
|
return new RedisWritePointDetector(index, allPatterns()).detect("DemoService.java", source);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static Set<String> allPatterns() {
|
||||||
|
return new HashSet<>(Arrays.asList("W01", "W02", "W03", "W04", "W05"));
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user