Compare commits
1 Commits
dev
...
e88408e9b6
| Author | SHA1 | Date | |
|---|---|---|---|
| e88408e9b6 |
@@ -4,7 +4,7 @@
|
|||||||
# 说明:
|
# 说明:
|
||||||
# - 本配置文件为业务覆盖配置,会与 jar 内 default-config.yaml 深度合并
|
# - 本配置文件为业务覆盖配置,会与 jar 内 default-config.yaml 深度合并
|
||||||
# - 未声明的项沿用工具内置默认值(忽略规则、检测模式等)
|
# - 未声明的项沿用工具内置默认值(忽略规则、检测模式等)
|
||||||
# - 当前以 Redis 缓存写入检测为主;MQ(RocketMQ/Kafka)方案见 docs/MQ序列化结构检测方案.md
|
# - 当前覆盖 Redis 缓存 + MQ(RocketMQ/Kafka)生产侧投递检测
|
||||||
|
|
||||||
# 总开关 true-执行检测 false-跳过检测(流水线直接通过,不发通知)
|
# 总开关 true-执行检测 false-跳过检测(流水线直接通过,不发通知)
|
||||||
enabled: true
|
enabled: true
|
||||||
@@ -33,3 +33,9 @@ manual_mappings:
|
|||||||
|
|
||||||
# 误报忽略(按需添加)
|
# 误报忽略(按需添加)
|
||||||
suppressions: []
|
suppressions: []
|
||||||
|
|
||||||
|
# MQ destination 忽略示例(按需取消注释)
|
||||||
|
# ignore:
|
||||||
|
# mq_destinations:
|
||||||
|
# - "benchmark:*"
|
||||||
|
# - "*:TEST"
|
||||||
|
|||||||
@@ -141,9 +141,10 @@ com/codechecker/serialization-schema-checker/1.0.0/
|
|||||||
| 无法获取 before / exit 2 | 浅克隆未取到对象、服务端禁 fetch SHA | 看日志中的 deepen;确认 Gitea 允许按 SHA fetch |
|
| 无法获取 before / exit 2 | 浅克隆未取到对象、服务端禁 fetch SHA | 看日志中的 deepen;确认 Gitea 允许按 SHA fetch |
|
||||||
| 下载 jar 失败 | Nexus 地址/版本错误 | 检查 env 变量 |
|
| 下载 jar 失败 | Nexus 地址/版本错误 | 检查 env 变量 |
|
||||||
| 未收到企微 | Secret 未配 / notify.enabled=false / webhook_url 空 | 检查配置 |
|
| 未收到企微 | Secret 未配 / notify.enabled=false / webhook_url 空 | 检查配置 |
|
||||||
| 大量误报 | 锁/计数器未过滤 | 补充 ignore.key_patterns |
|
| 大量 Redis 误报 | 锁/计数器未过滤 | 补充 ignore.key_patterns |
|
||||||
|
| 大量 MQ 误报 | 压测/临时 topic | 补充 ignore.mq_destinations |
|
||||||
| commit 数显示为 1(实际多个) | 浅克隆下 `rev-list` 看不到中间提交 | 已修复:优先事件 `commits` 长度;并 deepen 到 before 为祖先 |
|
| commit 数显示为 1(实际多个) | 浅克隆下 `rev-list` 看不到中间提交 | 已修复:优先事件 `commits` 长度;并 deepen 到 before 为祖先 |
|
||||||
| 漏报(模式/模块) | W0x 未开 / include_modules 过窄 | 确认 W01~W05;检查模块过滤 |
|
| 漏报(模式/模块) | W0x/MQ 未开 / include_modules 过窄 | 确认 W01~W05 与 mq_patterns;检查模块过滤 |
|
||||||
| 类型展开不完整 | 类型在依赖 jar 中 | 补充 `manual_mappings.value_type` |
|
| 类型展开不完整 | 类型在依赖 jar 中 | 补充 `manual_mappings.value_type` |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -1,8 +1,8 @@
|
|||||||
# MQ 消息体序列化结构变更检测 — 方案
|
# MQ 消息体序列化结构变更检测 — 方案
|
||||||
|
|
||||||
> 版本:v0.2
|
> 版本:v0.3
|
||||||
> 日期:2026-07-15
|
> 日期:2026-07-15
|
||||||
> 状态:**方案已落地(含 Kafka),开发未启动**
|
> 状态:**Phase M1 已实现**(MQ01/MQ02 + MQ-K01/MQ-K02);M2/M3 未启动
|
||||||
> 关联:复用 `serialization-schema-checker` 的 Schema 提取、Diff、企微通知与 CI 框架
|
> 关联:复用 `serialization-schema-checker` 的 Schema 提取、Diff、企微通知与 CI 框架
|
||||||
> 业务样本仓:`jnpf-java-cloud`(**RocketMQ + Kafka**)
|
> 业务样本仓:`jnpf-java-cloud`(**RocketMQ + Kafka**)
|
||||||
|
|
||||||
@@ -333,7 +333,7 @@ manual_mappings:
|
|||||||
|
|
||||||
## 10. 下一步
|
## 10. 下一步
|
||||||
|
|
||||||
1. 评审本方案(RocketMQ + Kafka 模式表)
|
1. ~~按 Phase M1 开发~~ **已完成**(MQ01/02 + MQ-K01/K02)
|
||||||
2. 按 **Phase M1** 开发(MQ01/02 + MQ-K01/K02)
|
2. Phase M2:MQ03~05、MQ-K03/K04、MQ-R 读侧补强
|
||||||
3. 回写 `配置说明.md` / `CI集成说明.md` 正式配置项
|
3. 业务仓验收 Topic:`capital-topic:WALLET_DEDUCT`(RocketMQ)、巡店/值班 Kafka topic
|
||||||
4. 验收 Topic 建议:`capital-topic:WALLET_DEDUCT`(RocketMQ)、巡店/值班 Kafka topic
|
4. 观察期按需扩大 `include_modules`
|
||||||
41
docs/配置说明.md
41
docs/配置说明.md
@@ -43,8 +43,11 @@ include_modules:
|
|||||||
由 `redisCheck` 仓库维护,随 jar 发布,默认包含:
|
由 `redisCheck` 仓库维护,随 jar 发布,默认包含:
|
||||||
|
|
||||||
- `detection.patterns`:**W01~W05**(JSON 字符串写入 + Template 直写 + Hash)
|
- `detection.patterns`:**W01~W05**(JSON 字符串写入 + Template 直写 + Hash)
|
||||||
|
- `detection.mq_patterns`:**MQ01/MQ02、MQ-K01/MQ-K02**(RocketMQ / Kafka 生产侧)
|
||||||
- `detection.read_hints_enabled`:W06 读侧反序列化类型辅助(默认 true)
|
- `detection.read_hints_enabled`:W06 读侧反序列化类型辅助(默认 true)
|
||||||
|
- `detection.mq_read_hints_enabled`:MQ 读侧补强(Phase M2,默认 false)
|
||||||
- `ignore.key_patterns`(锁 / 计数器 / token)
|
- `ignore.key_patterns`(锁 / 计数器 / token)
|
||||||
|
- `ignore.mq_destinations`(MQ destination 忽略,默认真空)
|
||||||
- `detection.min_confidence`、`max_field_depth`
|
- `detection.min_confidence`、`max_field_depth`
|
||||||
- `mode: notify`、`enabled: true`
|
- `mode: notify`、`enabled: true`
|
||||||
|
|
||||||
@@ -97,6 +100,9 @@ ignore:
|
|||||||
# 忽略的写入方法(类全名#方法名)
|
# 忽略的写入方法(类全名#方法名)
|
||||||
writer_methods: []
|
writer_methods: []
|
||||||
|
|
||||||
|
# 忽略的 MQ destination(topic 或 topic:tag,glob)
|
||||||
|
mq_destinations: []
|
||||||
|
|
||||||
# 检测规则
|
# 检测规则
|
||||||
detection:
|
detection:
|
||||||
# 启用的写入模式(默认已全部开启)
|
# 启用的写入模式(默认已全部开启)
|
||||||
@@ -107,9 +113,19 @@ detection:
|
|||||||
- W04 # redisTemplate 直写对象
|
- W04 # redisTemplate 直写对象
|
||||||
- W05 # opsForHash().put
|
- W05 # opsForHash().put
|
||||||
|
|
||||||
|
# MQ 生产侧投递(Phase M1)
|
||||||
|
mq_patterns:
|
||||||
|
- MQ01 # rocketMQTemplate.syncSend
|
||||||
|
- MQ02 # asyncSend / syncSendOrderly / sendOneWay
|
||||||
|
- MQ-K01 # kafkaTemplate.send(topic, payload)
|
||||||
|
- MQ-K02 # kafkaTemplate.send(topic, key, payload)
|
||||||
|
|
||||||
# W06:读侧反序列化类型辅助(不产生独立告警)
|
# W06:读侧反序列化类型辅助(不产生独立告警)
|
||||||
read_hints_enabled: true
|
read_hints_enabled: true
|
||||||
|
|
||||||
|
# MQ-R 读侧补强(Phase M2,默认关闭)
|
||||||
|
mq_read_hints_enabled: false
|
||||||
|
|
||||||
# 类型推断最低置信度,低于此值标记为低置信度提示
|
# 类型推断最低置信度,低于此值标记为低置信度提示
|
||||||
min_confidence: 0.6
|
min_confidence: 0.6
|
||||||
|
|
||||||
@@ -223,7 +239,8 @@ detection:
|
|||||||
|
|
||||||
- **不是写入模式**:不会单独因为「多了一处 parse」而告警
|
- **不是写入模式**:不会单独因为「多了一处 parse」而告警
|
||||||
- 能关联到 `redis get(key)` 时,还可补强 unresolved key
|
- 能关联到 `redis get(key)` 时,还可补强 unresolved key
|
||||||
- 覆盖优先级:`manual_mappings` > W06 > 写侧 AST
|
- 覆盖优先级:`manual_mappings` > W06 > 写侧 AST
|
||||||
|
- **仅补强 Redis 写入点**;MQ 投递点不走 W06(MQ-R 为 Phase M2)
|
||||||
|
|
||||||
关闭示例:
|
关闭示例:
|
||||||
|
|
||||||
@@ -232,6 +249,18 @@ detection:
|
|||||||
read_hints_enabled: false
|
read_hints_enabled: false
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### 3.5.2 detection.mq_patterns / ignore.mq_destinations(MQ Phase M1)
|
||||||
|
|
||||||
|
| 配置项 | 说明 |
|
||||||
|
|--------|------|
|
||||||
|
| `mq_patterns` | `MQ01`/`MQ02`(RocketMQ)、`MQ-K01`/`MQ-K02`(Kafka);默认已开启 |
|
||||||
|
| `mq_read_hints_enabled` | MQ-R 读侧补强,**默认 false** |
|
||||||
|
| `ignore.mq_destinations` | 忽略 destination(glob),如 `benchmark:*` |
|
||||||
|
|
||||||
|
企微 MQ 块:`- Topic --> ...`,并带 `> **通道**: RocketMQ|Kafka`。
|
||||||
|
|
||||||
|
详见:[MQ序列化结构检测方案.md](./MQ序列化结构检测方案.md)
|
||||||
|
|
||||||
### 3.6 manual_mappings
|
### 3.6 manual_mappings
|
||||||
|
|
||||||
当自动推断不准确时使用。匹配优先级 **高于** 自动推断(按 `类全名#方法名` 覆盖 key 模式与 value 类型)。
|
当自动推断不准确时使用。匹配优先级 **高于** 自动推断(按 `类全名#方法名` 覆盖 key 模式与 value 类型)。
|
||||||
@@ -346,10 +375,10 @@ include_modules: [] # 全仓
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 7. 扩展:MQ 消息体检测(方案阶段)
|
## 7. MQ 消息体检测(Phase M1 已落地)
|
||||||
|
|
||||||
MQ(**RocketMQ + Kafka**)消息体 Schema 变更监控方案已单独成文,**实现前不要求业务仓增配**。
|
同一 jar 默认启用 RocketMQ / Kafka **生产侧**投递检测(`detection.mq_patterns`),与 Redis 共用 Diff、骨架与企微模板。
|
||||||
|
|
||||||
详见:[MQ序列化结构检测方案.md](./MQ序列化结构检测方案.md)
|
- 配置说明见 §3.5.2
|
||||||
|
- 读侧 MQ-R、`MessageBuilder` / `ProducerRecord` 等见方案 Phase M2
|
||||||
届时预计新增:`detection.mq_patterns`(含 `MQ01~` 与 `MQ-K01~`)、`detection.mq_read_hints_enabled`、`ignore.mq_destinations`。
|
- 完整设计:[MQ序列化结构检测方案.md](./MQ序列化结构检测方案.md)
|
||||||
@@ -3,6 +3,7 @@ package com.codechecker.cache.analyze;
|
|||||||
import com.codechecker.cache.config.CheckerConfig;
|
import com.codechecker.cache.config.CheckerConfig;
|
||||||
import com.codechecker.cache.detector.CacheReadHint;
|
import com.codechecker.cache.detector.CacheReadHint;
|
||||||
import com.codechecker.cache.detector.CacheReadHintDetector;
|
import com.codechecker.cache.detector.CacheReadHintDetector;
|
||||||
|
import com.codechecker.cache.detector.MqWritePointDetector;
|
||||||
import com.codechecker.cache.detector.RedisWritePointDetector;
|
import com.codechecker.cache.detector.RedisWritePointDetector;
|
||||||
import com.codechecker.cache.detector.WritePoint;
|
import com.codechecker.cache.detector.WritePoint;
|
||||||
import com.codechecker.cache.diff.ChangeType;
|
import com.codechecker.cache.diff.ChangeType;
|
||||||
@@ -92,8 +93,13 @@ public class SchemaCheckAnalyzer {
|
|||||||
}
|
}
|
||||||
|
|
||||||
Set<String> patterns = new HashSet<>(config.getDetection().getPatterns());
|
Set<String> patterns = new HashSet<>(config.getDetection().getPatterns());
|
||||||
|
Set<String> mqPatterns = new HashSet<>(config.getDetection().getMqPatterns());
|
||||||
RedisWritePointDetector detectorNew = new RedisWritePointDetector(newIndex, patterns);
|
RedisWritePointDetector detectorNew = new RedisWritePointDetector(newIndex, patterns);
|
||||||
RedisWritePointDetector detectorOld = new RedisWritePointDetector(oldIndex, patterns);
|
RedisWritePointDetector detectorOld = new RedisWritePointDetector(oldIndex, patterns);
|
||||||
|
MqWritePointDetector mqDetectorNew = mqPatterns.isEmpty()
|
||||||
|
? null : new MqWritePointDetector(newIndex, mqPatterns);
|
||||||
|
MqWritePointDetector mqDetectorOld = mqPatterns.isEmpty()
|
||||||
|
? null : new MqWritePointDetector(oldIndex, mqPatterns);
|
||||||
JavaSchemaExtractor extractorNew = new JavaSchemaExtractor(newIndex, config.getDetection().getMaxFieldDepth());
|
JavaSchemaExtractor extractorNew = new JavaSchemaExtractor(newIndex, config.getDetection().getMaxFieldDepth());
|
||||||
JavaSchemaExtractor extractorOld = new JavaSchemaExtractor(oldIndex, config.getDetection().getMaxFieldDepth());
|
JavaSchemaExtractor extractorOld = new JavaSchemaExtractor(oldIndex, config.getDetection().getMaxFieldDepth());
|
||||||
SchemaDiffer differ = new SchemaDiffer();
|
SchemaDiffer differ = new SchemaDiffer();
|
||||||
@@ -110,9 +116,15 @@ public class SchemaCheckAnalyzer {
|
|||||||
boolean fileChanged = changedFiles.contains(path);
|
boolean fileChanged = changedFiles.contains(path);
|
||||||
String oldContent = fileChanged ? oldContents.get(path) : newContent;
|
String oldContent = fileChanged ? oldContents.get(path) : newContent;
|
||||||
|
|
||||||
List<WritePoint> newWps = detectorNew.detect(path, newContent);
|
List<WritePoint> newWps = new ArrayList<>(detectorNew.detect(path, newContent));
|
||||||
List<WritePoint> oldWps = oldContent == null
|
List<WritePoint> oldWps = oldContent == null
|
||||||
? new ArrayList<>() : detectorOld.detect(path, oldContent);
|
? new ArrayList<>() : new ArrayList<>(detectorOld.detect(path, oldContent));
|
||||||
|
if (mqDetectorNew != null) {
|
||||||
|
newWps.addAll(mqDetectorNew.detect(path, newContent));
|
||||||
|
}
|
||||||
|
if (mqDetectorOld != null && oldContent != null) {
|
||||||
|
oldWps.addAll(mqDetectorOld.detect(path, oldContent));
|
||||||
|
}
|
||||||
applyReadHints(newWps, path, newContent, newIndex);
|
applyReadHints(newWps, path, newContent, newIndex);
|
||||||
if (oldContent != null) {
|
if (oldContent != null) {
|
||||||
applyReadHints(oldWps, path, oldContent, oldIndex);
|
applyReadHints(oldWps, path, oldContent, oldIndex);
|
||||||
@@ -128,7 +140,7 @@ public class SchemaCheckAnalyzer {
|
|||||||
|
|
||||||
for (WritePoint nw : newWps) {
|
for (WritePoint nw : newWps) {
|
||||||
newSigs.add(nw.signature());
|
newSigs.add(nw.signature());
|
||||||
if (isKeyIgnored(nw.getResolvedKeyPattern()) || isWriterIgnored(nw)) {
|
if (isWritePointIgnored(nw) || isWriterIgnored(nw)) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
WritePoint ow = oldBySig.get(nw.signature());
|
WritePoint ow = oldBySig.get(nw.signature());
|
||||||
@@ -151,7 +163,9 @@ public class SchemaCheckAnalyzer {
|
|||||||
} else if (fileChanged) {
|
} else if (fileChanged) {
|
||||||
SchemaChange c = new SchemaChange(ChangeType.WRITE_POINT_ADDED);
|
SchemaChange c = new SchemaChange(ChangeType.WRITE_POINT_ADDED);
|
||||||
fillFromWritePoint(c, nw);
|
fillFromWritePoint(c, nw);
|
||||||
c.setMessage("新增缓存写入点,value 类型: " + displayType(nw));
|
c.setMessage(nw.isMq()
|
||||||
|
? "新增 MQ 投递点,value 类型: " + displayType(nw)
|
||||||
|
: "新增缓存写入点,value 类型: " + displayType(nw));
|
||||||
allChanges.add(c);
|
allChanges.add(c);
|
||||||
TypeSchema newSchema = extractorNew.extract(nw.getResolvedValueType(), nw.isRootArray());
|
TypeSchema newSchema = extractorNew.extract(nw.getResolvedValueType(), nw.isRootArray());
|
||||||
mergeKeyChange(keyChanges, nw,
|
mergeKeyChange(keyChanges, nw,
|
||||||
@@ -166,10 +180,12 @@ public class SchemaCheckAnalyzer {
|
|||||||
if (fileChanged) {
|
if (fileChanged) {
|
||||||
for (WritePoint ow : oldWps) {
|
for (WritePoint ow : oldWps) {
|
||||||
if (!newSigs.contains(ow.signature())
|
if (!newSigs.contains(ow.signature())
|
||||||
&& !isKeyIgnored(ow.getResolvedKeyPattern()) && !isWriterIgnored(ow)) {
|
&& !isWritePointIgnored(ow) && !isWriterIgnored(ow)) {
|
||||||
SchemaChange c = new SchemaChange(ChangeType.WRITE_POINT_REMOVED);
|
SchemaChange c = new SchemaChange(ChangeType.WRITE_POINT_REMOVED);
|
||||||
fillFromWritePoint(c, ow);
|
fillFromWritePoint(c, ow);
|
||||||
c.setMessage("删除缓存写入点,原 value 类型: " + displayType(ow));
|
c.setMessage(ow.isMq()
|
||||||
|
? "删除 MQ 投递点,原 value 类型: " + displayType(ow)
|
||||||
|
: "删除缓存写入点,原 value 类型: " + displayType(ow));
|
||||||
allChanges.add(c);
|
allChanges.add(c);
|
||||||
TypeSchema oldSchema = extractorOld.extract(ow.getResolvedValueType(), ow.isRootArray());
|
TypeSchema oldSchema = extractorOld.extract(ow.getResolvedValueType(), ow.isRootArray());
|
||||||
mergeKeyChange(keyChanges, ow,
|
mergeKeyChange(keyChanges, ow,
|
||||||
@@ -198,8 +214,12 @@ public class SchemaCheckAnalyzer {
|
|||||||
n.setWriteLocation(wp.location());
|
n.setWriteLocation(wp.location());
|
||||||
n.setValueType(displayType(wp));
|
n.setValueType(displayType(wp));
|
||||||
n.setKeyUnresolved(isUnresolvedKey(wp.getResolvedKeyPattern()));
|
n.setKeyUnresolved(isUnresolvedKey(wp.getResolvedKeyPattern()));
|
||||||
|
n.setChannel(wp.getChannel());
|
||||||
return n;
|
return n;
|
||||||
});
|
});
|
||||||
|
if (kc.getChannel() == null || kc.getChannel().isEmpty()) {
|
||||||
|
kc.setChannel(wp.getChannel());
|
||||||
|
}
|
||||||
if (kc.getWriteLocation() == null || kc.getWriteLocation().isEmpty()) {
|
if (kc.getWriteLocation() == null || kc.getWriteLocation().isEmpty()) {
|
||||||
kc.setWriteLocation(wp.location());
|
kc.setWriteLocation(wp.location());
|
||||||
}
|
}
|
||||||
@@ -229,11 +249,12 @@ public class SchemaCheckAnalyzer {
|
|||||||
|
|
||||||
/** 已解析 key 按模式聚合;未解析按「位置+表达式」拆分,避免串单。 */
|
/** 已解析 key 按模式聚合;未解析按「位置+表达式」拆分,避免串单。 */
|
||||||
private String aggregationKey(WritePoint wp) {
|
private String aggregationKey(WritePoint wp) {
|
||||||
|
String channel = wp.getChannel() == null ? WritePoint.CHANNEL_REDIS : wp.getChannel();
|
||||||
String pattern = wp.getResolvedKeyPattern();
|
String pattern = wp.getResolvedKeyPattern();
|
||||||
if (!isUnresolvedKey(pattern)) {
|
if (!isUnresolvedKey(pattern)) {
|
||||||
return pattern == null ? "<unknown>" : pattern;
|
return channel + "|" + (pattern == null ? "<unknown>" : pattern);
|
||||||
}
|
}
|
||||||
return "unknown|" + nvl(wp.location()) + "|" + nvl(wp.getKeyExpression());
|
return channel + "|unknown|" + nvl(wp.location()) + "|" + nvl(wp.getKeyExpression());
|
||||||
}
|
}
|
||||||
|
|
||||||
private boolean isUnresolvedKey(String keyPattern) {
|
private boolean isUnresolvedKey(String keyPattern) {
|
||||||
@@ -333,8 +354,14 @@ public class SchemaCheckAnalyzer {
|
|||||||
}
|
}
|
||||||
|
|
||||||
private String changeDedupKey(SchemaChange c) {
|
private String changeDedupKey(SchemaChange c) {
|
||||||
return c.getChangeType() + "|" + c.getKeyPattern() + "|"
|
ChangeType t = c.getChangeType();
|
||||||
+ c.getWriteLocation() + "|" + c.getFieldPath();
|
// 新增/删除投递点:按位置区分
|
||||||
|
if (t == ChangeType.WRITE_POINT_ADDED || t == ChangeType.WRITE_POINT_REMOVED) {
|
||||||
|
return t + "|" + c.getKeyPattern() + "|" + c.getWriteLocation();
|
||||||
|
}
|
||||||
|
// 同一 destination 上多处 send 同源 VO:字段级结构变更只保留一条
|
||||||
|
return t + "|" + c.getKeyPattern() + "|" + c.getFieldPath()
|
||||||
|
+ "|" + nvl(c.getOldValue()) + "|" + nvl(c.getNewValue());
|
||||||
}
|
}
|
||||||
|
|
||||||
private void enrich(List<SchemaChange> changes, WritePoint wp, double confidence) {
|
private void enrich(List<SchemaChange> changes, WritePoint wp, double confidence) {
|
||||||
@@ -358,9 +385,7 @@ public class SchemaCheckAnalyzer {
|
|||||||
if (isSuppressed(c)) {
|
if (isSuppressed(c)) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
String dedupKey = c.getChangeType() + "|" + c.getKeyPattern() + "|"
|
if (seen.add(changeDedupKey(c))) {
|
||||||
+ c.getWriteLocation() + "|" + c.getFieldPath();
|
|
||||||
if (seen.add(dedupKey)) {
|
|
||||||
result.add(c);
|
result.add(c);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -448,6 +473,9 @@ public class SchemaCheckAnalyzer {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
for (WritePoint wp : writePoints) {
|
for (WritePoint wp : writePoints) {
|
||||||
|
if (wp.isMq()) {
|
||||||
|
continue; // Redis W06 不补强 MQ 投递点(MQ-R 为 Phase M2)
|
||||||
|
}
|
||||||
enrichWritePointFromHints(wp, hints);
|
enrichWritePointFromHints(wp, hints);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -555,6 +583,13 @@ public class SchemaCheckAnalyzer {
|
|||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private boolean isWritePointIgnored(WritePoint wp) {
|
||||||
|
if (wp.isMq()) {
|
||||||
|
return isMqDestinationIgnored(wp.getResolvedKeyPattern());
|
||||||
|
}
|
||||||
|
return isKeyIgnored(wp.getResolvedKeyPattern());
|
||||||
|
}
|
||||||
|
|
||||||
private boolean isKeyIgnored(String keyPattern) {
|
private boolean isKeyIgnored(String keyPattern) {
|
||||||
if (keyPattern == null) {
|
if (keyPattern == null) {
|
||||||
return false;
|
return false;
|
||||||
@@ -567,6 +602,18 @@ public class SchemaCheckAnalyzer {
|
|||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private boolean isMqDestinationIgnored(String destination) {
|
||||||
|
if (destination == null) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
for (String glob : config.getIgnore().getMqDestinations()) {
|
||||||
|
if (GlobMatcher.matches(glob, destination)) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
private boolean isWriterIgnored(WritePoint wp) {
|
private boolean isWriterIgnored(WritePoint wp) {
|
||||||
String sig = wp.getEnclosingClass() + "#" + wp.getEnclosingMethod();
|
String sig = wp.getEnclosingClass() + "#" + wp.getEnclosingMethod();
|
||||||
return config.getIgnore().getWriterMethods().contains(sig);
|
return config.getIgnore().getWriterMethods().contains(sig);
|
||||||
|
|||||||
@@ -80,6 +80,8 @@ public class CheckerConfig {
|
|||||||
private List<String> keyPatterns = new ArrayList<>();
|
private List<String> keyPatterns = new ArrayList<>();
|
||||||
private List<String> filePatterns = new ArrayList<>();
|
private List<String> filePatterns = new ArrayList<>();
|
||||||
private List<String> writerMethods = new ArrayList<>();
|
private List<String> writerMethods = new ArrayList<>();
|
||||||
|
/** 忽略的 MQ destination 模式(topic / topic:tag) */
|
||||||
|
private List<String> mqDestinations = new ArrayList<>();
|
||||||
|
|
||||||
public List<String> getKeyPatterns() {
|
public List<String> getKeyPatterns() {
|
||||||
return keyPatterns;
|
return keyPatterns;
|
||||||
@@ -104,14 +106,26 @@ public class CheckerConfig {
|
|||||||
public void setWriterMethods(List<String> writerMethods) {
|
public void setWriterMethods(List<String> writerMethods) {
|
||||||
this.writerMethods = writerMethods;
|
this.writerMethods = writerMethods;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public List<String> getMqDestinations() {
|
||||||
|
return mqDestinations;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setMqDestinations(List<String> mqDestinations) {
|
||||||
|
this.mqDestinations = mqDestinations;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
public static class Detection {
|
public static class Detection {
|
||||||
private List<String> patterns = new ArrayList<>();
|
private List<String> patterns = new ArrayList<>();
|
||||||
|
/** MQ 投递检测模式:MQ01/MQ02/MQ-K01/MQ-K02… */
|
||||||
|
private List<String> mqPatterns = new ArrayList<>();
|
||||||
private double minConfidence = 0.6;
|
private double minConfidence = 0.6;
|
||||||
private int maxFieldDepth = 8;
|
private int maxFieldDepth = 8;
|
||||||
/** W06:是否启用读侧反序列化类型辅助补强 */
|
/** W06:是否启用读侧反序列化类型辅助补强 */
|
||||||
private boolean readHintsEnabled = true;
|
private boolean readHintsEnabled = true;
|
||||||
|
/** MQ-R:读侧 Listener / parse 补强(Phase M2;M1 默认 false) */
|
||||||
|
private boolean mqReadHintsEnabled = false;
|
||||||
|
|
||||||
public List<String> getPatterns() {
|
public List<String> getPatterns() {
|
||||||
return patterns;
|
return patterns;
|
||||||
@@ -121,6 +135,14 @@ public class CheckerConfig {
|
|||||||
this.patterns = patterns;
|
this.patterns = patterns;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public List<String> getMqPatterns() {
|
||||||
|
return mqPatterns;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setMqPatterns(List<String> mqPatterns) {
|
||||||
|
this.mqPatterns = mqPatterns;
|
||||||
|
}
|
||||||
|
|
||||||
public double getMinConfidence() {
|
public double getMinConfidence() {
|
||||||
return minConfidence;
|
return minConfidence;
|
||||||
}
|
}
|
||||||
@@ -144,6 +166,14 @@ public class CheckerConfig {
|
|||||||
public void setReadHintsEnabled(boolean readHintsEnabled) {
|
public void setReadHintsEnabled(boolean readHintsEnabled) {
|
||||||
this.readHintsEnabled = readHintsEnabled;
|
this.readHintsEnabled = readHintsEnabled;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public boolean isMqReadHintsEnabled() {
|
||||||
|
return mqReadHintsEnabled;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setMqReadHintsEnabled(boolean mqReadHintsEnabled) {
|
||||||
|
this.mqReadHintsEnabled = mqReadHintsEnabled;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
public static class ManualMapping {
|
public static class ManualMapping {
|
||||||
|
|||||||
@@ -94,13 +94,16 @@ public final class ConfigLoader {
|
|||||||
ig.setKeyPatterns(strList(ignore.get("key_patterns")));
|
ig.setKeyPatterns(strList(ignore.get("key_patterns")));
|
||||||
ig.setFilePatterns(strList(ignore.get("file_patterns")));
|
ig.setFilePatterns(strList(ignore.get("file_patterns")));
|
||||||
ig.setWriterMethods(strList(ignore.get("writer_methods")));
|
ig.setWriterMethods(strList(ignore.get("writer_methods")));
|
||||||
|
ig.setMqDestinations(strList(ignore.get("mq_destinations")));
|
||||||
|
|
||||||
Map<String, Object> detection = asMap(map.get("detection"));
|
Map<String, Object> detection = asMap(map.get("detection"));
|
||||||
CheckerConfig.Detection d = config.getDetection();
|
CheckerConfig.Detection d = config.getDetection();
|
||||||
d.setPatterns(strList(detection.get("patterns")));
|
d.setPatterns(strList(detection.get("patterns")));
|
||||||
|
d.setMqPatterns(strList(detection.get("mq_patterns")));
|
||||||
d.setMinConfidence(dbl(detection, "min_confidence", 0.6));
|
d.setMinConfidence(dbl(detection, "min_confidence", 0.6));
|
||||||
d.setMaxFieldDepth((int) lng(detection, "max_field_depth", 8));
|
d.setMaxFieldDepth((int) lng(detection, "max_field_depth", 8));
|
||||||
d.setReadHintsEnabled(bool(detection, "read_hints_enabled", true));
|
d.setReadHintsEnabled(bool(detection, "read_hints_enabled", true));
|
||||||
|
d.setMqReadHintsEnabled(bool(detection, "mq_read_hints_enabled", false));
|
||||||
|
|
||||||
Map<String, Object> severityOverrides = asMap(map.get("severity_overrides"));
|
Map<String, Object> severityOverrides = asMap(map.get("severity_overrides"));
|
||||||
Map<String, String> so = new LinkedHashMap<>();
|
Map<String, String> so = new LinkedHashMap<>();
|
||||||
|
|||||||
329
src/main/java/com/codechecker/cache/detector/MqWritePointDetector.java
vendored
Normal file
329
src/main/java/com/codechecker/cache/detector/MqWritePointDetector.java
vendored
Normal file
@@ -0,0 +1,329 @@
|
|||||||
|
package com.codechecker.cache.detector;
|
||||||
|
|
||||||
|
import com.codechecker.cache.key.RedisKeyResolver;
|
||||||
|
import com.codechecker.cache.schema.SourceIndex;
|
||||||
|
import com.github.javaparser.StaticJavaParser;
|
||||||
|
import com.github.javaparser.ast.CompilationUnit;
|
||||||
|
import com.github.javaparser.ast.body.CallableDeclaration;
|
||||||
|
import com.github.javaparser.ast.body.ClassOrInterfaceDeclaration;
|
||||||
|
import com.github.javaparser.ast.body.FieldDeclaration;
|
||||||
|
import com.github.javaparser.ast.body.MethodDeclaration;
|
||||||
|
import com.github.javaparser.ast.body.Parameter;
|
||||||
|
import com.github.javaparser.ast.body.VariableDeclarator;
|
||||||
|
import com.github.javaparser.ast.expr.BinaryExpr;
|
||||||
|
import com.github.javaparser.ast.expr.BooleanLiteralExpr;
|
||||||
|
import com.github.javaparser.ast.expr.CastExpr;
|
||||||
|
import com.github.javaparser.ast.expr.Expression;
|
||||||
|
import com.github.javaparser.ast.expr.IntegerLiteralExpr;
|
||||||
|
import com.github.javaparser.ast.expr.LongLiteralExpr;
|
||||||
|
import com.github.javaparser.ast.expr.MethodCallExpr;
|
||||||
|
import com.github.javaparser.ast.expr.NameExpr;
|
||||||
|
import com.github.javaparser.ast.expr.NullLiteralExpr;
|
||||||
|
import com.github.javaparser.ast.expr.ObjectCreationExpr;
|
||||||
|
import com.github.javaparser.ast.expr.StringLiteralExpr;
|
||||||
|
import com.github.javaparser.ast.type.ClassOrInterfaceType;
|
||||||
|
import com.github.javaparser.ast.type.Type;
|
||||||
|
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.Arrays;
|
||||||
|
import java.util.HashSet;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Optional;
|
||||||
|
import java.util.Set;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 检测 RocketMQ / Kafka 生产侧投递点(Phase M1:MQ01/MQ02、MQ-K01/MQ-K02)。
|
||||||
|
*/
|
||||||
|
public class MqWritePointDetector {
|
||||||
|
|
||||||
|
private static final Set<String> SERIALIZE_METHODS = new HashSet<>(Arrays.asList(
|
||||||
|
"toJSONString", "toJsonString", "getObjectToString", "toJsonStr", "writeValueAsString"));
|
||||||
|
private static final Set<String> ROCKET_SYNC = new HashSet<>(Arrays.asList("syncSend"));
|
||||||
|
private static final Set<String> ROCKET_ASYNC = new HashSet<>(Arrays.asList(
|
||||||
|
"asyncSend", "syncSendOrderly", "sendOneWay", "asyncSendOrderly"));
|
||||||
|
private static final Set<String> TRIVIAL_VALUE_CALLS = new HashSet<>(Arrays.asList(
|
||||||
|
"randomUUID", "toString", "valueOf"));
|
||||||
|
private static final Set<String> COLLECTION_SIMPLE = new HashSet<>(Arrays.asList(
|
||||||
|
"List", "ArrayList", "LinkedList", "Set", "HashSet", "Collection"));
|
||||||
|
private static final Set<String> IGNORE_PAYLOAD_TYPES = new HashSet<>(Arrays.asList(
|
||||||
|
"String", "byte", "Byte", "MessageExt", "Message", "ProducerRecord"));
|
||||||
|
|
||||||
|
private final SourceIndex index;
|
||||||
|
private final Set<String> enabledPatterns;
|
||||||
|
private final RedisKeyResolver keyResolver;
|
||||||
|
|
||||||
|
public MqWritePointDetector(SourceIndex index, Set<String> enabledPatterns) {
|
||||||
|
this.index = index;
|
||||||
|
this.enabledPatterns = enabledPatterns == null ? new HashSet<>() : enabledPatterns;
|
||||||
|
this.keyResolver = new RedisKeyResolver(index);
|
||||||
|
}
|
||||||
|
|
||||||
|
public List<WritePoint> detect(String filePath, String content) {
|
||||||
|
List<WritePoint> result = new ArrayList<>();
|
||||||
|
if (content == null || content.isEmpty() || enabledPatterns.isEmpty()) {
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
CompilationUnit cu;
|
||||||
|
try {
|
||||||
|
cu = StaticJavaParser.parse(content);
|
||||||
|
} catch (RuntimeException e) {
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
for (MethodCallExpr mce : cu.findAll(MethodCallExpr.class)) {
|
||||||
|
WritePoint wp = tryDetectRocketMq(mce, filePath);
|
||||||
|
if (wp == null) {
|
||||||
|
wp = tryDetectKafka(mce, filePath);
|
||||||
|
}
|
||||||
|
if (wp != null) {
|
||||||
|
result.add(wp);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
private WritePoint tryDetectRocketMq(MethodCallExpr mce, String filePath) {
|
||||||
|
String method = mce.getNameAsString();
|
||||||
|
String pattern = null;
|
||||||
|
if (ROCKET_SYNC.contains(method) && enabledPatterns.contains("MQ01")) {
|
||||||
|
pattern = "MQ01";
|
||||||
|
} else if (ROCKET_ASYNC.contains(method) && enabledPatterns.contains("MQ02")) {
|
||||||
|
pattern = "MQ02";
|
||||||
|
}
|
||||||
|
if (pattern == null || !isRocketMqScope(mce) || mce.getArguments().size() < 2) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
Expression destArg = mce.getArgument(0);
|
||||||
|
Expression payloadArg = mce.getArgument(1);
|
||||||
|
return buildMqWritePoint(mce, filePath, pattern, WritePoint.CHANNEL_ROCKETMQ,
|
||||||
|
destArg, payloadArg);
|
||||||
|
}
|
||||||
|
|
||||||
|
private WritePoint tryDetectKafka(MethodCallExpr mce, String filePath) {
|
||||||
|
if (!"send".equals(mce.getNameAsString()) || !isKafkaScope(mce)) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
int argc = mce.getArguments().size();
|
||||||
|
String pattern;
|
||||||
|
Expression topicArg;
|
||||||
|
Expression payloadArg;
|
||||||
|
if (argc == 2 && enabledPatterns.contains("MQ-K01")) {
|
||||||
|
pattern = "MQ-K01";
|
||||||
|
topicArg = mce.getArgument(0);
|
||||||
|
payloadArg = mce.getArgument(1);
|
||||||
|
} else if (argc >= 3 && enabledPatterns.contains("MQ-K02")) {
|
||||||
|
pattern = "MQ-K02";
|
||||||
|
topicArg = mce.getArgument(0);
|
||||||
|
payloadArg = mce.getArgument(argc - 1);
|
||||||
|
} else {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
return buildMqWritePoint(mce, filePath, pattern, WritePoint.CHANNEL_KAFKA,
|
||||||
|
topicArg, payloadArg);
|
||||||
|
}
|
||||||
|
|
||||||
|
private WritePoint buildMqWritePoint(MethodCallExpr mce, String filePath, String pattern,
|
||||||
|
String channel, Expression destArg, Expression payloadArg) {
|
||||||
|
Expression serialized = unwrapSerializer(payloadArg);
|
||||||
|
Expression typeExpr = serialized != null ? serialized : payloadArg;
|
||||||
|
if (serialized == null && isTrivialValue(payloadArg)) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
if (isBareStringOrBytesPayload(typeExpr, mce)) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
WritePoint wp = new WritePoint();
|
||||||
|
wp.setFilePath(filePath);
|
||||||
|
wp.setLineNumber(mce.getBegin().map(p -> p.line).orElse(0));
|
||||||
|
wp.setPattern(pattern);
|
||||||
|
wp.setChannel(channel);
|
||||||
|
wp.setKeyExpression(destArg.toString());
|
||||||
|
wp.setValueExpression(payloadArg.toString());
|
||||||
|
fillEnclosing(mce, wp);
|
||||||
|
|
||||||
|
SourceIndex.IndexedType context = index.get(wp.getEnclosingClass());
|
||||||
|
ClassOrInterfaceDeclaration enclosingDecl = mce
|
||||||
|
.findAncestor(ClassOrInterfaceDeclaration.class).orElse(null);
|
||||||
|
wp.setResolvedKeyPattern(keyResolver.resolve(destArg, enclosingDecl, context));
|
||||||
|
|
||||||
|
InferredType inferred = inferType(typeExpr, mce, context);
|
||||||
|
if (inferred != null) {
|
||||||
|
wp.setResolvedValueType(inferred.fqn);
|
||||||
|
wp.setRootArray(inferred.isArray);
|
||||||
|
wp.setConfidence(inferred.fqn == null ? 0.4 : 1.0);
|
||||||
|
} else {
|
||||||
|
wp.setConfidence(0.4);
|
||||||
|
}
|
||||||
|
return wp;
|
||||||
|
}
|
||||||
|
|
||||||
|
private boolean isRocketMqScope(MethodCallExpr mce) {
|
||||||
|
String scope = mce.getScope().map(Expression::toString).orElse("").toLowerCase();
|
||||||
|
return scope.contains("rocketmqtemplate") || scope.contains("rocketmq");
|
||||||
|
}
|
||||||
|
|
||||||
|
private boolean isKafkaScope(MethodCallExpr mce) {
|
||||||
|
String scope = mce.getScope().map(Expression::toString).orElse("").toLowerCase();
|
||||||
|
return scope.contains("kafkatemplate") || "kafkatemplate".equals(scope);
|
||||||
|
}
|
||||||
|
|
||||||
|
private boolean isBareStringOrBytesPayload(Expression typeExpr, MethodCallExpr contextCall) {
|
||||||
|
if (typeExpr instanceof StringLiteralExpr) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (typeExpr instanceof NameExpr) {
|
||||||
|
Type declared = findVariableType(((NameExpr) typeExpr).getNameAsString(), contextCall);
|
||||||
|
if (declared != null) {
|
||||||
|
String simple = declared.isClassOrInterfaceType()
|
||||||
|
? declared.asClassOrInterfaceType().getNameAsString()
|
||||||
|
: declared.asString();
|
||||||
|
if (IGNORE_PAYLOAD_TYPES.contains(simple) || "byte[]".equals(declared.asString())) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
private boolean isTrivialValue(Expression expr) {
|
||||||
|
if (expr instanceof StringLiteralExpr
|
||||||
|
|| expr instanceof IntegerLiteralExpr
|
||||||
|
|| expr instanceof LongLiteralExpr
|
||||||
|
|| expr instanceof BooleanLiteralExpr
|
||||||
|
|| expr instanceof NullLiteralExpr) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (expr instanceof BinaryExpr) {
|
||||||
|
BinaryExpr be = (BinaryExpr) expr;
|
||||||
|
if (be.getOperator() == BinaryExpr.Operator.PLUS) {
|
||||||
|
return isTrivialValue(be.getLeft()) && isTrivialValue(be.getRight());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (expr instanceof MethodCallExpr) {
|
||||||
|
MethodCallExpr call = (MethodCallExpr) expr;
|
||||||
|
String name = call.getNameAsString();
|
||||||
|
if (TRIVIAL_VALUE_CALLS.contains(name)) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if ("valueOf".equals(name) && !call.getArguments().isEmpty()) {
|
||||||
|
return isTrivialValue(call.getArgument(0));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (expr instanceof ObjectCreationExpr) {
|
||||||
|
return "UUID".equals(((ObjectCreationExpr) expr).getType().getNameAsString());
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
private Expression unwrapSerializer(Expression valueArg) {
|
||||||
|
if (valueArg instanceof MethodCallExpr) {
|
||||||
|
MethodCallExpr call = (MethodCallExpr) valueArg;
|
||||||
|
if (SERIALIZE_METHODS.contains(call.getNameAsString()) && !call.getArguments().isEmpty()) {
|
||||||
|
return call.getArgument(0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
private void fillEnclosing(MethodCallExpr mce, WritePoint wp) {
|
||||||
|
Optional<ClassOrInterfaceDeclaration> clazz = mce.findAncestor(ClassOrInterfaceDeclaration.class);
|
||||||
|
wp.setEnclosingClass(clazz.map(d -> d.getFullyQualifiedName().orElse(d.getNameAsString()))
|
||||||
|
.orElse("<unknown>"));
|
||||||
|
Optional<CallableDeclaration> method = mce.findAncestor(CallableDeclaration.class);
|
||||||
|
wp.setEnclosingMethod(method.map(CallableDeclaration::getNameAsString).orElse("<unknown>"));
|
||||||
|
}
|
||||||
|
|
||||||
|
private InferredType inferType(Expression expr, MethodCallExpr contextCall,
|
||||||
|
SourceIndex.IndexedType context) {
|
||||||
|
if (expr instanceof CastExpr) {
|
||||||
|
return resolveTypeNode(((CastExpr) expr).getType(), context);
|
||||||
|
}
|
||||||
|
if (expr instanceof ObjectCreationExpr) {
|
||||||
|
return resolveTypeNode(((ObjectCreationExpr) expr).getType(), context);
|
||||||
|
}
|
||||||
|
if (expr instanceof NameExpr) {
|
||||||
|
String name = ((NameExpr) expr).getNameAsString();
|
||||||
|
Type declared = findVariableType(name, contextCall);
|
||||||
|
if (declared != null) {
|
||||||
|
return resolveTypeNode(declared, context);
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
if (expr instanceof MethodCallExpr) {
|
||||||
|
MethodCallExpr call = (MethodCallExpr) expr;
|
||||||
|
Optional<ClassOrInterfaceDeclaration> clazz = contextCall
|
||||||
|
.findAncestor(ClassOrInterfaceDeclaration.class);
|
||||||
|
if (clazz.isPresent()) {
|
||||||
|
for (MethodDeclaration md : clazz.get().getMethods()) {
|
||||||
|
if (md.getNameAsString().equals(call.getNameAsString())) {
|
||||||
|
return resolveTypeNode(md.getType(), context);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
private Type findVariableType(String name, MethodCallExpr contextCall) {
|
||||||
|
Optional<CallableDeclaration> callable = contextCall.findAncestor(CallableDeclaration.class);
|
||||||
|
if (callable.isPresent()) {
|
||||||
|
CallableDeclaration<?> decl = callable.get();
|
||||||
|
for (VariableDeclarator var : decl.findAll(VariableDeclarator.class)) {
|
||||||
|
if (var.getNameAsString().equals(name)) {
|
||||||
|
return var.getType();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (Parameter p : decl.getParameters()) {
|
||||||
|
if (p.getNameAsString().equals(name)) {
|
||||||
|
return p.getType();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Optional<ClassOrInterfaceDeclaration> clazz = contextCall.findAncestor(ClassOrInterfaceDeclaration.class);
|
||||||
|
if (clazz.isPresent()) {
|
||||||
|
for (FieldDeclaration field : clazz.get().getFields()) {
|
||||||
|
for (VariableDeclarator var : field.getVariables()) {
|
||||||
|
if (var.getNameAsString().equals(name)) {
|
||||||
|
return var.getType();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
private InferredType resolveTypeNode(Type type, SourceIndex.IndexedType context) {
|
||||||
|
if (!(type instanceof ClassOrInterfaceType)) {
|
||||||
|
return new InferredType(null, false);
|
||||||
|
}
|
||||||
|
ClassOrInterfaceType cit = (ClassOrInterfaceType) type;
|
||||||
|
String simple = cit.getNameAsString();
|
||||||
|
if (COLLECTION_SIMPLE.contains(simple)) {
|
||||||
|
Optional<Type> arg = cit.getTypeArguments().filter(a -> !a.isEmpty()).map(a -> a.get(0));
|
||||||
|
if (arg.isPresent() && arg.get() instanceof ClassOrInterfaceType) {
|
||||||
|
String elementFqn = resolveFqn((ClassOrInterfaceType) arg.get(), context);
|
||||||
|
return new InferredType(elementFqn, true);
|
||||||
|
}
|
||||||
|
return new InferredType(null, true);
|
||||||
|
}
|
||||||
|
return new InferredType(resolveFqn(cit, context), false);
|
||||||
|
}
|
||||||
|
|
||||||
|
private String resolveFqn(ClassOrInterfaceType cit, SourceIndex.IndexedType context) {
|
||||||
|
String fqn = index.resolveFqn(cit.getNameWithScope(), context);
|
||||||
|
if (fqn == null) {
|
||||||
|
fqn = index.resolveFqn(cit.getNameAsString(), context);
|
||||||
|
}
|
||||||
|
return fqn;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static final class InferredType {
|
||||||
|
final String fqn;
|
||||||
|
final boolean isArray;
|
||||||
|
|
||||||
|
InferredType(String fqn, boolean isArray) {
|
||||||
|
this.fqn = fqn;
|
||||||
|
this.isArray = isArray;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,10 +1,14 @@
|
|||||||
package com.codechecker.cache.detector;
|
package com.codechecker.cache.detector;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 一个 Redis value 写入点的静态描述。
|
* 一个 Redis / MQ value 写入(投递)点的静态描述。
|
||||||
*/
|
*/
|
||||||
public class WritePoint {
|
public class WritePoint {
|
||||||
|
|
||||||
|
public static final String CHANNEL_REDIS = "REDIS";
|
||||||
|
public static final String CHANNEL_ROCKETMQ = "ROCKETMQ";
|
||||||
|
public static final String CHANNEL_KAFKA = "KAFKA";
|
||||||
|
|
||||||
private String filePath;
|
private String filePath;
|
||||||
private int lineNumber;
|
private int lineNumber;
|
||||||
private String enclosingClass;
|
private String enclosingClass;
|
||||||
@@ -20,6 +24,9 @@ public class WritePoint {
|
|||||||
|
|
||||||
private double confidence = 1.0;
|
private double confidence = 1.0;
|
||||||
|
|
||||||
|
/** REDIS / ROCKETMQ / KAFKA;默认 REDIS 兼容现网 */
|
||||||
|
private String channel = CHANNEL_REDIS;
|
||||||
|
|
||||||
/** 稳定标识:用于在 old/new 两个版本间配对同一写入点 */
|
/** 稳定标识:用于在 old/new 两个版本间配对同一写入点 */
|
||||||
public String signature() {
|
public String signature() {
|
||||||
return enclosingClass + "#" + enclosingMethod + "|" + normalizeKey();
|
return enclosingClass + "#" + enclosingMethod + "|" + normalizeKey();
|
||||||
@@ -29,6 +36,10 @@ public class WritePoint {
|
|||||||
return keyExpression == null ? "" : keyExpression.replaceAll("\\s+", "");
|
return keyExpression == null ? "" : keyExpression.replaceAll("\\s+", "");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public boolean isMq() {
|
||||||
|
return CHANNEL_ROCKETMQ.equals(channel) || CHANNEL_KAFKA.equals(channel);
|
||||||
|
}
|
||||||
|
|
||||||
public String getFilePath() {
|
public String getFilePath() {
|
||||||
return filePath;
|
return filePath;
|
||||||
}
|
}
|
||||||
@@ -117,6 +128,14 @@ public class WritePoint {
|
|||||||
this.confidence = confidence;
|
this.confidence = confidence;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public String getChannel() {
|
||||||
|
return channel;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setChannel(String channel) {
|
||||||
|
this.channel = channel == null || channel.isEmpty() ? CHANNEL_REDIS : channel;
|
||||||
|
}
|
||||||
|
|
||||||
public String location() {
|
public String location() {
|
||||||
String simpleClass = enclosingClass;
|
String simpleClass = enclosingClass;
|
||||||
if (simpleClass != null && simpleClass.contains(".")) {
|
if (simpleClass != null && simpleClass.contains(".")) {
|
||||||
|
|||||||
@@ -7,19 +7,21 @@ import java.util.ArrayList;
|
|||||||
import java.util.List;
|
import java.util.List;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 按 Redis key(或未知 key 时的写入点)聚合后的结构变更摘要。
|
* 按 Redis key / MQ Topic(或未知时的写入点)聚合后的结构变更摘要。
|
||||||
*/
|
*/
|
||||||
public class KeyStructureChange {
|
public class KeyStructureChange {
|
||||||
|
|
||||||
/** 解析到的 key 模式;未知时为 unknown-key */
|
/** 解析到的 key / destination 模式;未知时为 unknown-key */
|
||||||
private String keyPattern;
|
private String keyPattern;
|
||||||
/** 源码中的 key 表达式,如 req.getKey() */
|
/** 源码中的 key / destination 表达式 */
|
||||||
private String keyExpression;
|
private String keyExpression;
|
||||||
/** 写入位置 Class#method:line */
|
/** 写入位置 Class#method:line */
|
||||||
private String writeLocation;
|
private String writeLocation;
|
||||||
/** 展示用 value 类型,如 List<ClockInExportVo> */
|
/** 展示用 value 类型,如 List<ClockInExportVo> */
|
||||||
private String valueType;
|
private String valueType;
|
||||||
private boolean keyUnresolved;
|
private boolean keyUnresolved;
|
||||||
|
/** REDIS / ROCKETMQ / KAFKA */
|
||||||
|
private String channel = "REDIS";
|
||||||
private String oldSkeletonJson;
|
private String oldSkeletonJson;
|
||||||
private String newSkeletonJson;
|
private String newSkeletonJson;
|
||||||
private Severity severity = Severity.P2;
|
private Severity severity = Severity.P2;
|
||||||
@@ -65,6 +67,28 @@ public class KeyStructureChange {
|
|||||||
this.keyUnresolved = keyUnresolved;
|
this.keyUnresolved = keyUnresolved;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public String getChannel() {
|
||||||
|
return channel;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setChannel(String channel) {
|
||||||
|
this.channel = channel == null || channel.isEmpty() ? "REDIS" : channel;
|
||||||
|
}
|
||||||
|
|
||||||
|
public boolean isMq() {
|
||||||
|
return "ROCKETMQ".equals(channel) || "KAFKA".equals(channel);
|
||||||
|
}
|
||||||
|
|
||||||
|
public String channelDisplay() {
|
||||||
|
if ("ROCKETMQ".equals(channel)) {
|
||||||
|
return "RocketMQ";
|
||||||
|
}
|
||||||
|
if ("KAFKA".equals(channel)) {
|
||||||
|
return "Kafka";
|
||||||
|
}
|
||||||
|
return "Redis";
|
||||||
|
}
|
||||||
|
|
||||||
public String getOldSkeletonJson() {
|
public String getOldSkeletonJson() {
|
||||||
return oldSkeletonJson;
|
return oldSkeletonJson;
|
||||||
}
|
}
|
||||||
@@ -102,7 +126,7 @@ public class KeyStructureChange {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/** 通知里 Key 行展示文本:未解析优先用表达式。 */
|
/** 通知里 Key/Topic 行展示文本:未解析优先用表达式。 */
|
||||||
public String displayKey() {
|
public String displayKey() {
|
||||||
if (keyUnresolved && keyExpression != null && !keyExpression.trim().isEmpty()) {
|
if (keyUnresolved && keyExpression != null && !keyExpression.trim().isEmpty()) {
|
||||||
return keyExpression.trim();
|
return keyExpression.trim();
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ import java.nio.charset.StandardCharsets;
|
|||||||
import java.util.ArrayList;
|
import java.util.ArrayList;
|
||||||
import java.util.Collections;
|
import java.util.Collections;
|
||||||
import java.util.EnumMap;
|
import java.util.EnumMap;
|
||||||
|
import java.util.LinkedHashSet;
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
import java.util.Map;
|
import java.util.Map;
|
||||||
import java.util.Set;
|
import java.util.Set;
|
||||||
@@ -124,7 +125,10 @@ public class ReportBuilder {
|
|||||||
|
|
||||||
private String renderKeyBlock(KeyStructureChange kc) {
|
private String renderKeyBlock(KeyStructureChange kc) {
|
||||||
StringBuilder sb = new StringBuilder();
|
StringBuilder sb = new StringBuilder();
|
||||||
appendKeyLine(sb, kc.displayKey(), kc.getKeyExpression(), kc.isKeyUnresolved());
|
appendDestinationLine(sb, kc);
|
||||||
|
if (kc.isMq()) {
|
||||||
|
sb.append(" > **通道**: ").append(kc.channelDisplay()).append('\n');
|
||||||
|
}
|
||||||
appendMetaLines(sb, kc.getWriteLocation(), kc.getValueType());
|
appendMetaLines(sb, kc.getWriteLocation(), kc.getValueType());
|
||||||
String oldJson = nvl(kc.getOldSkeletonJson());
|
String oldJson = nvl(kc.getOldSkeletonJson());
|
||||||
String newJson = nvl(kc.getNewSkeletonJson());
|
String newJson = nvl(kc.getNewSkeletonJson());
|
||||||
@@ -142,7 +146,8 @@ public class ReportBuilder {
|
|||||||
if (oldJson.isEmpty() && !newJson.isEmpty()) {
|
if (oldJson.isEmpty() && !newJson.isEmpty()) {
|
||||||
sb.append(" > **value 新增为:** ").append(newRendered).append('\n');
|
sb.append(" > **value 新增为:** ").append(newRendered).append('\n');
|
||||||
} else if (!oldJson.isEmpty() && newJson.isEmpty()) {
|
} else if (!oldJson.isEmpty() && newJson.isEmpty()) {
|
||||||
sb.append(" > **value 原结构:** ").append(oldRendered).append("(已删除写入)\n");
|
sb.append(" > **value 原结构:** ").append(oldRendered)
|
||||||
|
.append(kc.isMq() ? "(已删除投递)\n" : "(已删除写入)\n");
|
||||||
} else {
|
} else {
|
||||||
sb.append(" > **value值由:** ").append(oldRendered).append('\n');
|
sb.append(" > **value值由:** ").append(oldRendered).append('\n');
|
||||||
sb.append(" > **变更为:** ").append(newRendered).append('\n');
|
sb.append(" > **变更为:** ").append(newRendered).append('\n');
|
||||||
@@ -161,6 +166,7 @@ public class ReportBuilder {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
List<String> parts = new ArrayList<>();
|
List<String> parts = new ArrayList<>();
|
||||||
|
Set<String> seen = new LinkedHashSet<>();
|
||||||
for (SchemaChange c : details) {
|
for (SchemaChange c : details) {
|
||||||
if (c == null || c.getChangeType() != ChangeType.TYPE_CHANGED) {
|
if (c == null || c.getChangeType() != ChangeType.TYPE_CHANGED) {
|
||||||
continue;
|
continue;
|
||||||
@@ -171,6 +177,10 @@ public class ReportBuilder {
|
|||||||
}
|
}
|
||||||
String oldType = displayJavaType(c.getOldValue());
|
String oldType = displayJavaType(c.getOldValue());
|
||||||
String newType = displayJavaType(c.getNewValue());
|
String newType = displayJavaType(c.getNewValue());
|
||||||
|
String dedup = path + "|" + oldType + "|" + newType;
|
||||||
|
if (!seen.add(dedup)) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
StringBuilder part = new StringBuilder();
|
StringBuilder part = new StringBuilder();
|
||||||
// 字段名用普通文本(避免反引号被企微渲染成色块)
|
// 字段名用普通文本(避免反引号被企微渲染成色块)
|
||||||
part.append(path);
|
part.append(path);
|
||||||
@@ -213,8 +223,29 @@ public class ReportBuilder {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Key 行:展示解析后的模式或未解析时的源码表达式;未解析时追加灰色提示。
|
* Key / Topic 行:展示解析后的模式或未解析时的源码表达式;未解析时追加灰色提示。
|
||||||
* 反引号内仍须转义 {@code *},否则企微会把 {@code *:*} 当成斜体吃掉通配符。
|
*/
|
||||||
|
private void appendDestinationLine(StringBuilder sb, KeyStructureChange kc) {
|
||||||
|
boolean mq = kc.isMq();
|
||||||
|
String label = mq ? "Topic -->" : "Key -->";
|
||||||
|
String keyText = kc.isKeyUnresolved() && kc.getKeyExpression() != null
|
||||||
|
&& !kc.getKeyExpression().trim().isEmpty()
|
||||||
|
? kc.getKeyExpression().trim()
|
||||||
|
: nvl(kc.displayKey());
|
||||||
|
if (keyText.isEmpty()) {
|
||||||
|
keyText = "unknown-key";
|
||||||
|
}
|
||||||
|
sb.append("- ").append(label).append(" `").append(escapeWeComCode(keyText)).append('`');
|
||||||
|
if (kc.isKeyUnresolved()) {
|
||||||
|
sb.append(mq
|
||||||
|
? " <font color=\"comment\">(destination 未解析)</font>"
|
||||||
|
: " <font color=\"comment\">(key 无法解析)</font>");
|
||||||
|
}
|
||||||
|
sb.append('\n');
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Key 行(无 KeyStructureChange 时的退路)。
|
||||||
*/
|
*/
|
||||||
private void appendKeyLine(StringBuilder sb, String displayKey, String keyExpression,
|
private void appendKeyLine(StringBuilder sb, String displayKey, String keyExpression,
|
||||||
boolean unresolved) {
|
boolean unresolved) {
|
||||||
|
|||||||
@@ -35,6 +35,8 @@ ignore:
|
|||||||
- "**/test/**"
|
- "**/test/**"
|
||||||
# 忽略的写入方法(类全名#方法名)
|
# 忽略的写入方法(类全名#方法名)
|
||||||
writer_methods: []
|
writer_methods: []
|
||||||
|
# 忽略的 MQ destination(topic 或 topic:tag,glob)
|
||||||
|
mq_destinations: []
|
||||||
|
|
||||||
# 检测规则
|
# 检测规则
|
||||||
detection:
|
detection:
|
||||||
@@ -44,8 +46,16 @@ detection:
|
|||||||
- W03 # stringRedisTemplate.opsForValue().set(key, JsonUtil.getObjectToString(x), ...)
|
- W03 # stringRedisTemplate.opsForValue().set(key, JsonUtil.getObjectToString(x), ...)
|
||||||
- W04 # redisTemplate.opsForValue().set(key, obj, ...)
|
- W04 # redisTemplate.opsForValue().set(key, obj, ...)
|
||||||
- W05 # redisTemplate.opsForHash().put(key, field, obj)
|
- W05 # redisTemplate.opsForHash().put(key, field, obj)
|
||||||
|
# MQ 生产侧投递检测(Phase M1)
|
||||||
|
mq_patterns:
|
||||||
|
- MQ01 # rocketMQTemplate.syncSend(dest, payload)
|
||||||
|
- MQ02 # asyncSend / syncSendOrderly / sendOneWay
|
||||||
|
- MQ-K01 # kafkaTemplate.send(topic, payload)
|
||||||
|
- MQ-K02 # kafkaTemplate.send(topic, key, payload)
|
||||||
# W06 读侧辅助:用 parseObject / getJsonToBean 等补强写入点 value 类型(非写入模式)
|
# W06 读侧辅助:用 parseObject / getJsonToBean 等补强写入点 value 类型(非写入模式)
|
||||||
read_hints_enabled: true
|
read_hints_enabled: true
|
||||||
|
# MQ-R 读侧补强(Phase M2,默认关闭)
|
||||||
|
mq_read_hints_enabled: false
|
||||||
# 类型推断最低置信度,低于此值标记为低置信度提示
|
# 类型推断最低置信度,低于此值标记为低置信度提示
|
||||||
min_confidence: 0.6
|
min_confidence: 0.6
|
||||||
# 字段展开最大深度(防止循环引用)
|
# 字段展开最大深度(防止循环引用)
|
||||||
|
|||||||
123
src/test/java/com/codechecker/cache/MqScenarioTest.java
vendored
Normal file
123
src/test/java/com/codechecker/cache/MqScenarioTest.java
vendored
Normal file
@@ -0,0 +1,123 @@
|
|||||||
|
package com.codechecker.cache;
|
||||||
|
|
||||||
|
import com.codechecker.cache.detector.MqWritePointDetector;
|
||||||
|
import com.codechecker.cache.detector.WritePoint;
|
||||||
|
import com.codechecker.cache.diff.ChangeType;
|
||||||
|
import com.codechecker.cache.diff.SchemaChange;
|
||||||
|
import com.codechecker.cache.diff.SchemaDiffer;
|
||||||
|
import com.codechecker.cache.report.CheckReport;
|
||||||
|
import com.codechecker.cache.report.KeyStructureChange;
|
||||||
|
import com.codechecker.cache.report.ReportBuilder;
|
||||||
|
import com.codechecker.cache.schema.JavaSchemaExtractor;
|
||||||
|
import com.codechecker.cache.schema.SourceIndex;
|
||||||
|
import com.codechecker.cache.schema.TypeSchema;
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
|
||||||
|
import java.util.HashSet;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Set;
|
||||||
|
import java.util.stream.Collectors;
|
||||||
|
|
||||||
|
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||||
|
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||||
|
|
||||||
|
class MqScenarioTest {
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void rocketMqFieldRemovalProducesTopicDiff() {
|
||||||
|
String constants = TestSupport.fixture("fixtures/mq/rocket-wallet/CapitalMqConstants.txt");
|
||||||
|
String oldReq = TestSupport.fixture("fixtures/mq/rocket-wallet/WalletDeductReqOld.txt");
|
||||||
|
String newReq = TestSupport.fixture("fixtures/mq/rocket-wallet/WalletDeductReqNew.txt");
|
||||||
|
String producer = TestSupport.fixture("fixtures/mq/rocket-wallet/WalletDeductProducer.txt");
|
||||||
|
|
||||||
|
SourceIndex oldIndex = new SourceIndex();
|
||||||
|
oldIndex.addSource(constants);
|
||||||
|
oldIndex.addSource(oldReq);
|
||||||
|
oldIndex.addSource(producer);
|
||||||
|
|
||||||
|
SourceIndex newIndex = new SourceIndex();
|
||||||
|
newIndex.addSource(constants);
|
||||||
|
newIndex.addSource(newReq);
|
||||||
|
newIndex.addSource(producer);
|
||||||
|
|
||||||
|
Set<String> patterns = new HashSet<>();
|
||||||
|
patterns.add("MQ01");
|
||||||
|
WritePoint wp = new MqWritePointDetector(newIndex, patterns)
|
||||||
|
.detect("WalletDeductProducer.java", producer).get(0);
|
||||||
|
|
||||||
|
TypeSchema oldSchema = new JavaSchemaExtractor(oldIndex, 8)
|
||||||
|
.extract(wp.getResolvedValueType(), wp.isRootArray());
|
||||||
|
TypeSchema newSchema = new JavaSchemaExtractor(newIndex, 8)
|
||||||
|
.extract(wp.getResolvedValueType(), wp.isRootArray());
|
||||||
|
List<SchemaChange> changes = new SchemaDiffer().diff(oldSchema, newSchema);
|
||||||
|
List<ChangeType> types = changes.stream().map(SchemaChange::getChangeType).collect(Collectors.toList());
|
||||||
|
assertTrue(types.contains(ChangeType.FIELD_REMOVED), "应删除 remark 字段");
|
||||||
|
|
||||||
|
CheckReport report = new CheckReport();
|
||||||
|
report.setRepository("demo");
|
||||||
|
report.setBranch("feature/mq");
|
||||||
|
report.setOldSha("aaa");
|
||||||
|
report.setNewSha("bbb");
|
||||||
|
KeyStructureChange key = new KeyStructureChange();
|
||||||
|
key.setKeyPattern(wp.getResolvedKeyPattern());
|
||||||
|
key.setChannel(WritePoint.CHANNEL_ROCKETMQ);
|
||||||
|
key.setWriteLocation(wp.location());
|
||||||
|
key.setValueType("WalletDeductReq");
|
||||||
|
key.setOldSkeletonJson("{\"walletId\":\"\",\"amount\":0,\"remark\":\"\"}");
|
||||||
|
key.setNewSkeletonJson("{\"walletId\":\"\",\"amount\":0}");
|
||||||
|
key.getFieldDetails().addAll(changes);
|
||||||
|
report.getKeyChanges().add(key);
|
||||||
|
report.getChanges().addAll(changes);
|
||||||
|
|
||||||
|
String md = new ReportBuilder("[序列化结构变更]").toMarkdown(report);
|
||||||
|
assertTrue(md.contains("- Topic --> `capital-topic:WALLET_DEDUCT`"), md);
|
||||||
|
assertTrue(md.contains("> **通道**: RocketMQ"), md);
|
||||||
|
assertTrue(md.contains("<font color=\"warning\">\"remark\":\"\"</font>")
|
||||||
|
|| md.contains("remark"), md);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void kafkaListFieldRemovalProducesTopicDiff() {
|
||||||
|
String oldVo = TestSupport.fixture("fixtures/mq/kafka-patrol/CheckItemDetailVoOld.txt");
|
||||||
|
String newVo = TestSupport.fixture("fixtures/mq/kafka-patrol/CheckItemDetailVoNew.txt");
|
||||||
|
String service = TestSupport.fixture("fixtures/mq/kafka-patrol/PatrolService.txt");
|
||||||
|
|
||||||
|
SourceIndex oldIndex = new SourceIndex();
|
||||||
|
oldIndex.addSource(oldVo);
|
||||||
|
oldIndex.addSource(service);
|
||||||
|
SourceIndex newIndex = new SourceIndex();
|
||||||
|
newIndex.addSource(newVo);
|
||||||
|
newIndex.addSource(service);
|
||||||
|
|
||||||
|
Set<String> patterns = new HashSet<>();
|
||||||
|
patterns.add("MQ-K01");
|
||||||
|
WritePoint wp = new MqWritePointDetector(newIndex, patterns)
|
||||||
|
.detect("PatrolService.java", service).get(0);
|
||||||
|
assertEquals("patrol-store-food-safe:*", wp.getResolvedKeyPattern());
|
||||||
|
assertTrue(wp.isRootArray());
|
||||||
|
|
||||||
|
TypeSchema oldSchema = new JavaSchemaExtractor(oldIndex, 8)
|
||||||
|
.extract(wp.getResolvedValueType(), true);
|
||||||
|
TypeSchema newSchema = new JavaSchemaExtractor(newIndex, 8)
|
||||||
|
.extract(wp.getResolvedValueType(), true);
|
||||||
|
List<SchemaChange> changes = new SchemaDiffer().diff(oldSchema, newSchema);
|
||||||
|
assertTrue(changes.stream().anyMatch(c -> c.getChangeType() == ChangeType.FIELD_REMOVED));
|
||||||
|
|
||||||
|
CheckReport report = new CheckReport();
|
||||||
|
report.setRepository("demo");
|
||||||
|
KeyStructureChange key = new KeyStructureChange();
|
||||||
|
key.setKeyPattern(wp.getResolvedKeyPattern());
|
||||||
|
key.setChannel(WritePoint.CHANNEL_KAFKA);
|
||||||
|
key.setValueType("List<CheckItemDetailVo>");
|
||||||
|
key.setWriteLocation(wp.location());
|
||||||
|
key.setOldSkeletonJson("[{\"itemId\":\"\",\"itemName\":\"\",\"score\":0}]");
|
||||||
|
key.setNewSkeletonJson("[{\"itemId\":\"\",\"itemName\":\"\"}]");
|
||||||
|
key.getFieldDetails().addAll(changes);
|
||||||
|
report.getKeyChanges().add(key);
|
||||||
|
|
||||||
|
String md = new ReportBuilder("[序列化结构变更]").toMarkdown(report);
|
||||||
|
assertTrue(md.contains("- Topic --> `patrol-store-food-safe:*`"), md);
|
||||||
|
assertTrue(md.contains("> **通道**: Kafka"), md);
|
||||||
|
assertTrue(md.contains("> **类型**: List<CheckItemDetailVo>"), md);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -19,6 +19,9 @@ class ConfigLoaderTest {
|
|||||||
assertTrue(config.isEnabled());
|
assertTrue(config.isEnabled());
|
||||||
assertEquals("notify", config.getMode());
|
assertEquals("notify", config.getMode());
|
||||||
assertTrue(config.getDetection().getPatterns().contains("W01"));
|
assertTrue(config.getDetection().getPatterns().contains("W01"));
|
||||||
|
assertTrue(config.getDetection().getMqPatterns().contains("MQ01"));
|
||||||
|
assertTrue(config.getDetection().getMqPatterns().contains("MQ-K01"));
|
||||||
|
assertFalse(config.getDetection().isMqReadHintsEnabled());
|
||||||
assertFalse(config.isScanTestSources());
|
assertFalse(config.isScanTestSources());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
66
src/test/java/com/codechecker/cache/detector/MqWritePointDetectorTest.java
vendored
Normal file
66
src/test/java/com/codechecker/cache/detector/MqWritePointDetectorTest.java
vendored
Normal file
@@ -0,0 +1,66 @@
|
|||||||
|
package com.codechecker.cache.detector;
|
||||||
|
|
||||||
|
import com.codechecker.cache.TestSupport;
|
||||||
|
import com.codechecker.cache.schema.SourceIndex;
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
|
||||||
|
import java.util.HashSet;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Set;
|
||||||
|
|
||||||
|
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||||
|
import static org.junit.jupiter.api.Assertions.assertFalse;
|
||||||
|
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||||
|
|
||||||
|
class MqWritePointDetectorTest {
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void detectsRocketMqSyncSendWithTopicTag() {
|
||||||
|
String constants = TestSupport.fixture("fixtures/mq/rocket-wallet/CapitalMqConstants.txt");
|
||||||
|
String req = TestSupport.fixture("fixtures/mq/rocket-wallet/WalletDeductReqOld.txt");
|
||||||
|
String producer = TestSupport.fixture("fixtures/mq/rocket-wallet/WalletDeductProducer.txt");
|
||||||
|
|
||||||
|
SourceIndex index = new SourceIndex();
|
||||||
|
index.addSource(constants);
|
||||||
|
index.addSource(req);
|
||||||
|
index.addSource(producer);
|
||||||
|
|
||||||
|
Set<String> patterns = new HashSet<>();
|
||||||
|
patterns.add("MQ01");
|
||||||
|
patterns.add("MQ02");
|
||||||
|
List<WritePoint> wps = new MqWritePointDetector(index, patterns)
|
||||||
|
.detect("WalletDeductProducer.java", producer);
|
||||||
|
|
||||||
|
assertEquals(1, wps.size());
|
||||||
|
WritePoint wp = wps.get(0);
|
||||||
|
assertEquals("MQ01", wp.getPattern());
|
||||||
|
assertEquals(WritePoint.CHANNEL_ROCKETMQ, wp.getChannel());
|
||||||
|
assertEquals("capital-topic:WALLET_DEDUCT", wp.getResolvedKeyPattern());
|
||||||
|
assertTrue(wp.getResolvedValueType().endsWith("WalletDeductReq"));
|
||||||
|
assertFalse(wp.isRootArray());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void detectsKafkaSendListAsRootArray() {
|
||||||
|
String vo = TestSupport.fixture("fixtures/mq/kafka-patrol/CheckItemDetailVoOld.txt");
|
||||||
|
String service = TestSupport.fixture("fixtures/mq/kafka-patrol/PatrolService.txt");
|
||||||
|
|
||||||
|
SourceIndex index = new SourceIndex();
|
||||||
|
index.addSource(vo);
|
||||||
|
index.addSource(service);
|
||||||
|
|
||||||
|
Set<String> patterns = new HashSet<>();
|
||||||
|
patterns.add("MQ-K01");
|
||||||
|
patterns.add("MQ-K02");
|
||||||
|
List<WritePoint> wps = new MqWritePointDetector(index, patterns)
|
||||||
|
.detect("PatrolService.java", service);
|
||||||
|
|
||||||
|
assertEquals(1, wps.size());
|
||||||
|
WritePoint wp = wps.get(0);
|
||||||
|
assertEquals("MQ-K01", wp.getPattern());
|
||||||
|
assertEquals(WritePoint.CHANNEL_KAFKA, wp.getChannel());
|
||||||
|
assertEquals("patrol-store-food-safe:*", wp.getResolvedKeyPattern());
|
||||||
|
assertTrue(wp.getResolvedValueType().endsWith("CheckItemDetailVo"));
|
||||||
|
assertTrue(wp.isRootArray());
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -168,6 +168,37 @@ class ReportBuilderTest {
|
|||||||
"应包含类型变更摘要行,实际 markdown:\n" + md);
|
"应包含类型变更摘要行,实际 markdown:\n" + md);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void typeChangeSummaryDedupesSameFieldFromMultipleWrites() {
|
||||||
|
CheckReport report = baseReport();
|
||||||
|
KeyStructureChange key = new KeyStructureChange();
|
||||||
|
key.setKeyPattern("data-analysis-task-topic");
|
||||||
|
key.setChannel("ROCKETMQ");
|
||||||
|
key.setWriteLocation("FtbDataAnalysisTaskServiceImpl#createTasks:48");
|
||||||
|
key.setValueType("DataAnalysisBO");
|
||||||
|
key.setOldSkeletonJson("{\"test1\":\"\"}");
|
||||||
|
key.setNewSkeletonJson("{\"test1\":0}");
|
||||||
|
|
||||||
|
SchemaChange a = new SchemaChange(ChangeType.TYPE_CHANGED);
|
||||||
|
a.setFieldPath("test1");
|
||||||
|
a.setOldValue("String/STRING");
|
||||||
|
a.setNewValue("Integer/NUMBER");
|
||||||
|
a.setWriteLocation("FtbDataAnalysisTaskServiceImpl#createTasks:48");
|
||||||
|
SchemaChange b = new SchemaChange(ChangeType.TYPE_CHANGED);
|
||||||
|
b.setFieldPath("test1");
|
||||||
|
b.setOldValue("String/STRING");
|
||||||
|
b.setNewValue("Integer/NUMBER");
|
||||||
|
b.setWriteLocation("FtbDataAnalysisTaskServiceImpl#createTasks:52");
|
||||||
|
key.getFieldDetails().add(a);
|
||||||
|
key.getFieldDetails().add(b);
|
||||||
|
report.getKeyChanges().add(key);
|
||||||
|
|
||||||
|
String md = new ReportBuilder("[序列化结构变更]").toMarkdown(report);
|
||||||
|
assertTrue(md.contains("> **类型变更**: test1 <font color=\"warning\">String → Integer</font>"), md);
|
||||||
|
assertFalse(md.contains("String → Integer</font>;test1"),
|
||||||
|
"同字段类型变更不应重复,实际:\n" + md);
|
||||||
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
void wrapperAddedHighlightedBeforeNestedMovedFields() {
|
void wrapperAddedHighlightedBeforeNestedMovedFields() {
|
||||||
SchemaChange wrapper = new SchemaChange(ChangeType.WRAPPER_ADDED);
|
SchemaChange wrapper = new SchemaChange(ChangeType.WRAPPER_ADDED);
|
||||||
|
|||||||
Reference in New Issue
Block a user