Compare commits
9 Commits
fd5aab5e7f
...
dev-1.0-MQ
| Author | SHA1 | Date | |
|---|---|---|---|
| bef70dd032 | |||
| 4361fe767f | |||
| bb406cd354 | |||
| d884aa36ff | |||
| e88408e9b6 | |||
| 14f9f1fce7 | |||
| 4dd4944107 | |||
| a03e1b4819 | |||
| 27dfeb8a4c |
@@ -4,7 +4,7 @@
|
||||
# 说明:
|
||||
# - 本配置文件为业务覆盖配置,会与 jar 内 default-config.yaml 深度合并
|
||||
# - 未声明的项沿用工具内置默认值(忽略规则、检测模式等)
|
||||
# - 当前以 Redis 缓存写入检测为主;MQ(RocketMQ/Kafka)方案见 docs/MQ序列化结构检测方案.md
|
||||
# - 当前覆盖 Redis 缓存 + MQ(RocketMQ/Kafka)生产侧投递与 MQ-R 读侧补强
|
||||
|
||||
# 总开关 true-执行检测 false-跳过检测(流水线直接通过,不发通知)
|
||||
enabled: true
|
||||
@@ -33,3 +33,9 @@ manual_mappings:
|
||||
|
||||
# 误报忽略(按需添加)
|
||||
suppressions: []
|
||||
|
||||
# MQ destination 忽略示例(按需取消注释)
|
||||
# ignore:
|
||||
# mq_destinations:
|
||||
# - "benchmark:*"
|
||||
# - "*:TEST"
|
||||
|
||||
@@ -15,7 +15,7 @@ Gitea Actions 触发
|
||||
↓
|
||||
java -jar 执行(对比 before → after,累计 diff)
|
||||
↓
|
||||
有结构变更 → 企微通知(按 Key 骨架;删除橙/新增绿)
|
||||
有结构变更 → 企微通知(按 Key 骨架;删除橙/新增绿/修改灰)
|
||||
↓
|
||||
mode=block 且含任意结构变更 → exit 1(流水线失败)
|
||||
```
|
||||
@@ -95,18 +95,18 @@ java -jar serialization-schema-checker-1.0.0.jar \
|
||||
## 5. 与现有流水线的关系
|
||||
|
||||
| 流水线 | 作用 | 关系 |
|
||||
|--------|------|------|
|
||||
|--------|----------------|------|
|
||||
| `demo.yaml` (AI代码质量分析) | AI Code Review | 并行,互不影响 |
|
||||
| `code-check` (CodeChecker) | 通用变更检测 | **同模式**,可并列执行 |
|
||||
| `serialization-schema-check` | 缓存结构检测 | 新增 |
|
||||
| `serialization-schema-check` | 序列化结构检测 | 新增 |
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 6. 工具发布流程(redisCheck 仓库)
|
||||
## 6. 工具发布流程(schemaCheck 仓库)
|
||||
|
||||
```bash
|
||||
# 在 redisCheck 仓库根目录
|
||||
# 在 schemaCheck 仓库根目录
|
||||
mvn clean package -DskipTests
|
||||
|
||||
# 发布到 Nexus(需配置 settings.xml)
|
||||
@@ -141,9 +141,10 @@ com/codechecker/serialization-schema-checker/1.0.0/
|
||||
| 无法获取 before / exit 2 | 浅克隆未取到对象、服务端禁 fetch SHA | 看日志中的 deepen;确认 Gitea 允许按 SHA fetch |
|
||||
| 下载 jar 失败 | Nexus 地址/版本错误 | 检查 env 变量 |
|
||||
| 未收到企微 | Secret 未配 / notify.enabled=false / webhook_url 空 | 检查配置 |
|
||||
| 大量误报 | 锁/计数器未过滤 | 补充 ignore.key_patterns |
|
||||
| 大量 Redis 误报 | 锁/计数器未过滤 | 补充 ignore.key_patterns |
|
||||
| 大量 MQ 误报 | 压测/临时 topic | 补充 ignore.mq_destinations |
|
||||
| commit 数显示为 1(实际多个) | 浅克隆下 `rev-list` 看不到中间提交 | 已修复:优先事件 `commits` 长度;并 deepen 到 before 为祖先 |
|
||||
| 漏报(模式/模块) | W0x 未开 / include_modules 过窄 | 确认 W01~W05;检查模块过滤 |
|
||||
| 漏报(模式/模块) | W0x/MQ 未开 / include_modules 过窄 | 确认 W01~W05 与 mq_patterns(含 MQ03~05、MQ-K03/K04);检查 `mq_read_hints_enabled` 与模块过滤 |
|
||||
| 类型展开不完整 | 类型在依赖 jar 中 | 补充 `manual_mappings.value_type` |
|
||||
|
||||
---
|
||||
@@ -176,6 +177,6 @@ java -jar /path/to/serialization-schema-checker-1.0.0.jar \
|
||||
|
||||
| 文档 | 说明 |
|
||||
|------|------|
|
||||
| [实施方案.md](./实施方案.md) | 缓存检测总体方案 |
|
||||
| [实施方案.md](redis序列化结构检测实施方案.md) | 缓存检测总体方案 |
|
||||
| [配置说明.md](./配置说明.md) | YAML 配置项 |
|
||||
| [MQ序列化结构检测方案.md](./MQ序列化结构检测方案.md) | MQ 消息体 Schema(RocketMQ + Kafka,方案已落地) |
|
||||
| [MQ序列化结构检测方案.md](MQ序列化结构检测实施方案.md) | MQ 消息体 Schema(RocketMQ + Kafka,方案已落地) |
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# MQ 消息体序列化结构变更检测 — 方案
|
||||
|
||||
> 版本:v0.2
|
||||
> 版本:v0.4
|
||||
> 日期:2026-07-15
|
||||
> 状态:**方案已落地(含 Kafka),开发未启动**
|
||||
> 状态:**Phase M1 + M2 已实现**(生产侧 MQ01~05 / MQ-K01~K04 + 读侧 MQ-R)
|
||||
> 关联:复用 `serialization-schema-checker` 的 Schema 提取、Diff、企微通知与 CI 框架
|
||||
> 业务样本仓:`jnpf-java-cloud`(**RocketMQ + Kafka**)
|
||||
|
||||
@@ -37,7 +37,7 @@
|
||||
|
||||
在 push 时静态分析 **消息体类型的序列化 Schema** 是否相对对比区间发生变更,覆盖 **RocketMQ + Kafka**,并复用现有企微通知 / notify|block 能力。
|
||||
|
||||
### 1.3 非目标(本方案首版)
|
||||
### 1.3 非目标
|
||||
|
||||
- 不连接真实 Broker,不拉取积压消息做运行时校验
|
||||
- 不解析依赖 jar 内消息类型(仅本仓 `src/main/java`)
|
||||
@@ -56,8 +56,9 @@
|
||||
| `rocketMQTemplate.syncSend(dest, dto)` | 高(如钱包扣费) | **纳入** |
|
||||
| `asyncSend` / `syncSendOrderly` 等 | 中 | **纳入** |
|
||||
| `convertAndSend` | 视封装而定 | **纳入** |
|
||||
| 先 `JSON.toJSONString` 再发 String | 较低 | unwrap 后取类型 |
|
||||
| 只发 `String` / `byte[]` / `MessageExt` | 有 | **默认忽略** |
|
||||
| `MessageBuilder.withPayload` 再 send | 中(如 IM 延时消息) | **纳入**(MQ04) |
|
||||
| 先 `JSON.toJSONString` 再发 String | 较低 | unwrap 后取类型(MQ05) |
|
||||
| 只发 `String` / `byte[]` / 无泛型 `Message` | 有 | **默认忽略** |
|
||||
|
||||
样本(资金钱包):
|
||||
|
||||
@@ -68,12 +69,14 @@ rocketMQTemplate.syncSend(CapitalMqConstants.TOPIC + ":" + tag, req); // WalletD
|
||||
public class WalletDeductConsumer implements RocketMQListener<WalletDeductReq> { ... }
|
||||
```
|
||||
|
||||
### 2.2 Kafka(已确认需纳入)
|
||||
### 2.2 Kafka
|
||||
|
||||
| 写法 | 出现情况 | 策略 |
|
||||
|------|----------|------|
|
||||
| `kafkaTemplate.send(topic, dto)` | 中(值班食安项等) | **纳入** |
|
||||
| `kafkaTemplate.send(topic, List<Xxx>)` | 中(巡店食安项列表) | **纳入**(rootArray) |
|
||||
| `kafkaTemplate.send(ProducerRecord)` | 中 | **纳入**(MQ-K03) |
|
||||
| 先 JSON 再 `send(topic, json)` | 较低 | unwrap(MQ-K04) |
|
||||
| `@KafkaListener` + `String` + `JSONObject.parseObject(..., Xxx.class)` | 有(数据分析中差评) | **读侧补强 MQ-R** |
|
||||
| `KafkaTopicUtil` 仅创建 Topic | 有(租户) | **忽略**(无消息体) |
|
||||
|
||||
@@ -102,7 +105,7 @@ public void handleMessage(String message) {
|
||||
|
||||
动态 Topic(如按租户拼接)静态推断结果形如 `patrol-store-topic:*`,与 Redis key `*` 规则一致。
|
||||
|
||||
### 2.3 「Key」等价物(destination)
|
||||
### 2.3 「Key」等价物
|
||||
|
||||
| 中间件 | 聚合键形态 | 来源 |
|
||||
|--------|------------|------|
|
||||
@@ -111,13 +114,15 @@ public void handleMessage(String message) {
|
||||
|
||||
无法解析时:展示表达式 + `<font color="comment">(destination 未解析)</font>`。
|
||||
|
||||
### 2.4 读侧补强(类比 W06)
|
||||
### 2.4 读侧补强
|
||||
|
||||
| 中间件 | 补强来源 |
|
||||
|--------|----------|
|
||||
| RocketMQ | `RocketMQListener<T>`、`onMessage(T)` + `@RocketMQMessageListener` |
|
||||
| Kafka | `@KafkaListener` 方法参数类型;或方法内 `parseObject/parseArray(..., Xxx.class)`(与现有 W06 共享解析能力) |
|
||||
|
||||
开关:`detection.mq_read_hints_enabled`(默认 **true**)。
|
||||
|
||||
---
|
||||
|
||||
## 3. 方案总览
|
||||
@@ -135,8 +140,8 @@ public void handleMessage(String message) {
|
||||
```text
|
||||
Git Diff → 变更 Java 文件
|
||||
├─ Redis:W01~W05 + W06 ← 已有
|
||||
└─ MQ:RocketMQ(MQ01~)+ Kafka(MQ-K*)
|
||||
+ Listener / parse 补强(MQ-R) ← 本方案
|
||||
└─ MQ:RocketMQ(MQ01~05)+ Kafka(MQ-K01~K04)
|
||||
+ Listener / parse 补强(MQ-R) ← 已实现
|
||||
↓
|
||||
同一套 TypeSchema / SchemaDiffer / Skeleton / WeCom
|
||||
```
|
||||
@@ -161,7 +166,7 @@ Git Diff → 变更 Java 文件
|
||||
| MQ01 | `rocketMQTemplate.syncSend(dest, payload, …)` | dest、payload 类型 |
|
||||
| MQ02 | `asyncSend` / `syncSendOrderly` / `sendOneWay` 等 | 同上 |
|
||||
| MQ03 | `convertAndSend(dest, payload)` | 同上 |
|
||||
| MQ04 | `MessageBuilder.withPayload(obj)` 再 send | payload 类型 |
|
||||
| MQ04 | `MessageBuilder.withPayload(obj)` 再 send,或 `Message<T>` | payload / `T` |
|
||||
| MQ05 | 先 `toJSONString`/`getObjectToString` 再 send String | unwrap 后类型 |
|
||||
|
||||
### 4.2 生产侧 — Kafka
|
||||
@@ -170,7 +175,7 @@ Git Diff → 变更 Java 文件
|
||||
|---------|------------|------|
|
||||
| MQ-K01 | `kafkaTemplate.send(topic, payload)` | topic、payload 类型 |
|
||||
| MQ-K02 | `kafkaTemplate.send(topic, key, payload)` | 同上(忽略分区 key) |
|
||||
| MQ-K03 | `send(ProducerRecord)` / `ListenableFuture` 封装若可解析 | topic + value 类型 |
|
||||
| MQ-K03 | `send(ProducerRecord)` | topic + value 类型 |
|
||||
| MQ-K04 | 先 JSON 序列化为 String 再 `send(topic, json)` | unwrap 后类型 |
|
||||
|
||||
Payload 为 `List<Xxx>` / `Collection` 时标记 **rootArray**,骨架为 JSON 数组(与 Redis List 一致)。
|
||||
@@ -187,7 +192,7 @@ Payload 为 `List<Xxx>` / `Collection` 时标记 **rootArray**,骨架为 JSON
|
||||
|---------|------|------|
|
||||
| MQ-R01 | `RocketMQListener<T>` / `@RocketMQMessageListener` | 补强同 destination 生产点 |
|
||||
| MQ-R02 | `@KafkaListener` + 参数类型 `T`(非 String) | 补强同 topic |
|
||||
| MQ-R03 | Listener 内 `parseObject`/`parseArray(..., Xxx.class)` | 补强(可复用 W06 检测器) |
|
||||
| MQ-R03 | Listener 内 `parseObject`/`parseArray(..., Xxx.class)` | 补强(复用 parse AST) |
|
||||
|
||||
开关:`detection.mq_read_hints_enabled`(默认 true)。
|
||||
|
||||
@@ -220,7 +225,6 @@ Payload 为 `List<Xxx>` / `Collection` 时标记 **rootArray**,骨架为 JSON
|
||||
|
||||
- 删除字段橙 `warning`;新增绿 `info`
|
||||
- destination 未解析时灰色提示
|
||||
- 可选后缀:「请评估消费积压与兼容反序列化」
|
||||
|
||||
「通道」字段用于区分中间件;若模板求简,可省略通道仅靠 Topic 形态区分。
|
||||
|
||||
@@ -272,37 +276,36 @@ manual_mappings:
|
||||
|
||||
## 7. 分阶段交付
|
||||
|
||||
### Phase M1 — MVP(RocketMQ + Kafka 基础投递)
|
||||
### Phase M1 — MVP(RocketMQ + Kafka 基础投递)✅
|
||||
|
||||
| 任务 | 说明 |
|
||||
|------|------|
|
||||
| MQ01/MQ02 | RocketMQ `syncSend` / `asyncSend` |
|
||||
| MQ-K01/MQ-K02 | Kafka `send(topic, payload)` / 三参 send |
|
||||
| destination 推断 | 字面量、常量、拼接;Kafka 动态 topic → `*` |
|
||||
| Schema Diff + 骨架通知 | 复用 ReportBuilder;可选「通道」行 |
|
||||
| Schema Diff + 骨架通知 | 复用 ReportBuilder;「通道」行 |
|
||||
| 夹具 | `fixtures/mq/rocket-wallet/`、`fixtures/mq/kafka-patrol/` |
|
||||
| 配置 | `mq_patterns`(含 MQ-K*)、`ignore.mq_destinations` |
|
||||
| List 根数组骨架 | `List<CheckItemDetailVo>` 等(与 M2 需求合并交付) |
|
||||
|
||||
**验收**:
|
||||
|
||||
1. 删 `WalletDeductReq` 字段 → 企微出现 RocketMQ Topic 骨架变更
|
||||
2. 删 `CheckItemDetailVo` 字段 → 企微出现 Kafka Topic 骨架变更
|
||||
|
||||
### Phase M2 — 增强
|
||||
### Phase M2 — 增强 ✅
|
||||
|
||||
| 任务 | 说明 |
|
||||
|------|------|
|
||||
| MQ03~MQ05、MQ-K03/K04 | convertAndSend、MessageBuilder、JSON 字符串发送、ProducerRecord |
|
||||
| MQ03~MQ05、MQ-K03/K04 | convertAndSend、MessageBuilder/`Message<T>`、JSON 字符串发送、ProducerRecord |
|
||||
| MQ-R01~R03 | RocketMQ Listener + Kafka `@KafkaListener` / parse 补强 |
|
||||
| List 根数组骨架 | 巡店 `List<CheckItemDetailVo>` 等 |
|
||||
| 夹具 | `fixtures/mq/rocket-im/`、`fixtures/mq/kafka-record/` |
|
||||
|
||||
### Phase M3 — 运营
|
||||
**验收**:
|
||||
|
||||
| 任务 | 说明 |
|
||||
|------|------|
|
||||
| 积压风险提示文案 | 统一提示评估消费 lag / 积压 |
|
||||
| 更多夹具 | 值班 Kafka、中差评 Listener、IM Favorite 等 |
|
||||
| 分 webhook / 标题前缀 | 缓存 vs MQ 可选拆分 |
|
||||
1. `MessageBuilder.withPayload(DutyImNotice)` + `asyncSend` → 命中 MQ04,改 VO 字段可告警
|
||||
2. `kafkaTemplate.send(ProducerRecord)` → 命中 MQ-K03
|
||||
3. Listener / `parseObject` 可补强同 Topic 弱类型生产点
|
||||
|
||||
---
|
||||
|
||||
@@ -312,7 +315,7 @@ manual_mappings:
|
||||
|------|------|
|
||||
| Kafka topic 按租户动态拼接 | 归一 `prefix:*`;`manual_mappings` |
|
||||
| RocketMQ / Kafka 混用同一 VO | 各投递点独立告警(符合预期) |
|
||||
| Listener 收 String、parse 在方法深处 | MQ-R03 + 复用 W06 AST |
|
||||
| Listener 收 String、parse 在方法深处 | MQ-R03 + parse AST |
|
||||
| 生产/消费跨模块对不齐 | 同仓索引 + destination 对齐;失败则仅写侧 |
|
||||
| Jackson / Fastjson / Kafka JsonSerializer 细节差 | 首版字段名;必要时方言或 mapping |
|
||||
| 只改消费未改生产类型 | 不告警(工具职责是消息体 Schema) |
|
||||
@@ -323,17 +326,14 @@ manual_mappings:
|
||||
|
||||
| 项 | 结论 |
|
||||
|----|------|
|
||||
| 中间件范围 | **RocketMQ + Kafka**(本仓已确认);Rabbit 暂不纳入 |
|
||||
| 中间件范围 | **RocketMQ + Kafka**|
|
||||
| 对比区间 | `gitea.event.before` → `gitea.sha` |
|
||||
| 阻断 | 与现网 `mode` 共用 |
|
||||
| 级别 | 不引入 P0/P1/P2 产品展示 |
|
||||
| 交付 | 同一 jar;patterns 区分 Redis / MQ(含 MQ-K*) |
|
||||
|
||||
---
|
||||
|
||||
## 10. 下一步
|
||||
|
||||
1. 评审本方案(RocketMQ + Kafka 模式表)
|
||||
2. 按 **Phase M1** 开发(MQ01/02 + MQ-K01/K02)
|
||||
3. 回写 `配置说明.md` / `CI集成说明.md` 正式配置项
|
||||
4. 验收 Topic 建议:`capital-topic:WALLET_DEDUCT`(RocketMQ)、巡店/值班 Kafka topic
|
||||
1. ~~按 Phase M1 开发~~ **已完成**(MQ01/02 + MQ-K01/K02)
|
||||
2. ~~按 Phase M2 开发~~ **已完成**(MQ03~05、MQ-K03/K04、MQ-R)
|
||||
3. 业务仓验收:`MessageBuilder` IM Topic、巡店/值班 Kafka、`capital-topic:WALLET_DEDUCT`
|
||||
@@ -4,7 +4,7 @@
|
||||
> 日期:2026-07-15
|
||||
> 技术栈:Java 11 + Maven + JavaParser
|
||||
> 目标仓库:`schemaCheck`(工具) / `jnpf-java-cloud`(被检测业务仓库)
|
||||
> 当前阶段:**Phase 1 + Phase 2 已完成**;Phase 3 运营待做;**MQ 扩展方案已文档落地**(见 `docs/MQ序列化结构检测方案.md`)
|
||||
> 当前阶段:**Phase 1 + Phase 2 已完成**;**MQ 扩展方案已文档落地**(见 `docs/MQ序列化结构检测方案.md`)
|
||||
|
||||
---
|
||||
|
||||
@@ -161,13 +161,11 @@ flowchart TB
|
||||
| 报告 | Jackson | 2.15.x | JSON/Markdown 报告序列化 |
|
||||
| 测试 | JUnit 5 | 5.10.x | 单元测试 + 夹具样本 |
|
||||
|
||||
**不采用** Spoon / Eclipse JDT 的原因:JavaParser 足够覆盖第一版需求,依赖更轻,CLI 启动更快。
|
||||
|
||||
**Lombok 处理策略**:基于源码字段 + `@Data` 等注解推断序列化字段;对 `@Builder`、`@SuperBuilder` 等复杂场景标记为低置信度提示。后续可选集成 `lombok.ast` 或 delombok 预处理。
|
||||
|
||||
---
|
||||
|
||||
## 5. 工程结构(redisCheck 仓库)
|
||||
## 5. 工程结构(schemaCheck 仓库)
|
||||
|
||||
```text
|
||||
schemaCheck/
|
||||
@@ -527,31 +525,6 @@ notify:
|
||||
| 企微高亮 | 删除橙 `warning` / 新增绿 `info`;位置+类型通用项 | ✅ |
|
||||
| W06 读侧辅助 | `parseObject`/`parseArray` 等补强写入 value 类型 | ✅ |
|
||||
|
||||
### Phase 3 — 运营
|
||||
|
||||
| 任务 | 说明 |
|
||||
|------|------|
|
||||
| 报告落盘 | 可选输出 JSON 报告文件 |
|
||||
| 误报反馈 | `suppressions` 按写入点 / change_types 精细忽略 |
|
||||
| 更多业务场景覆盖 | 考勤、文件下载进度等 |
|
||||
|
||||
### Phase 4 — MQ 消息体结构检测(方案已落地,含 Kafka,开发待启)
|
||||
|
||||
业务仓同时存在:
|
||||
|
||||
- **RocketMQ**:`RocketMQTemplate.syncSend(topic:tag, dto)` / `RocketMQListener<T>`
|
||||
- **Kafka**:`KafkaTemplate.send(topic, vo|List)` / `@KafkaListener` + `parseObject`
|
||||
|
||||
消息体字段变更会导致积压旧消息反序列化失败,风险模型与 Redis 同类。
|
||||
|
||||
| 任务 | 说明 | 状态 |
|
||||
|------|------|------|
|
||||
| 方案文档 | RocketMQ + Kafka 模式、destination、复用 Schema Diff/企微 | ✅ 见专用文档 |
|
||||
| Phase M1 | RocketMQ syncSend + Kafka send + 骨架通知 + 夹具 | 待启 |
|
||||
| Phase M2/M3 | Listener/parse 补强、更多投递形态、运营 | 待启 |
|
||||
|
||||
**专用方案:** [`docs/MQ序列化结构检测方案.md`](./MQ序列化结构检测方案.md)
|
||||
|
||||
---
|
||||
|
||||
## 11. 测试策略
|
||||
@@ -561,6 +534,8 @@ notify:
|
||||
- `SchemaDifferTest`:纯字段路径对比逻辑
|
||||
- `JavaSchemaExtractorTest`:类字段展开、注解、内部类
|
||||
- `RedisWritePointDetectorTest`:各种写入 AST 模式匹配
|
||||
- `MqWritePointDetectorTest`:RocketMQ/Kafka 投递(含 MQ04 MessageBuilder、MQ-K03 ProducerRecord)
|
||||
- `MqReadHintDetectorTest`:MQ-R Listener / parseObject 补强
|
||||
- `RedisKeyResolverTest`:常量、format、拼接推断
|
||||
|
||||
### 11.2 样本夹具测试(fixtures)
|
||||
@@ -574,6 +549,10 @@ notify:
|
||||
| `fixtures/tenant/` | 包装结构变更(TenantVO → CacheEnvelope) |
|
||||
| `fixtures/lock/` | 锁/计数器/token 应被忽略 |
|
||||
| `fixtures/template/` | W04 Template 直写 |
|
||||
| `fixtures/mq/rocket-wallet/` | RocketMQ syncSend(MQ01) |
|
||||
| `fixtures/mq/rocket-im/` | MessageBuilder + asyncSend(MQ04)、convertAndSend(MQ03) |
|
||||
| `fixtures/mq/kafka-patrol/` | Kafka List 根数组(MQ-K01) |
|
||||
| `fixtures/mq/kafka-record/` | ProducerRecord(MQ-K03)、JSON 字符串 send(MQ-K04) |
|
||||
|
||||
### 11.3 端到端测试
|
||||
|
||||
@@ -605,7 +584,7 @@ notify:
|
||||
| 4 | 上线策略 | 先 `notify` ,稳定后手动切 `block` |
|
||||
| 5 | 检测范围 | **仅 `src/main/java`**,不扫描测试代码 |
|
||||
|
||||
以上决策已纳入实施方案;**Phase 1 / Phase 2 已交付**。缓存侧可进入 Phase 3;MQ 侧以 [`MQ序列化结构检测方案.md`](./MQ序列化结构检测方案.md) 为准评审后开发。
|
||||
以上决策已纳入实施方案;**Phase 1 / Phase 2 已交付**。缓存侧可进入 Phase 3;MQ 侧以 [`MQ序列化结构检测方案.md`](MQ序列化结构检测实施方案.md) 为准评审后开发。
|
||||
|
||||
相关文档:
|
||||
|
||||
42
docs/配置说明.md
42
docs/配置说明.md
@@ -43,8 +43,11 @@ include_modules:
|
||||
由 `redisCheck` 仓库维护,随 jar 发布,默认包含:
|
||||
|
||||
- `detection.patterns`:**W01~W05**(JSON 字符串写入 + Template 直写 + Hash)
|
||||
- `detection.mq_patterns`:**MQ01~MQ05、MQ-K01~MQ-K04**(RocketMQ / Kafka 生产侧)
|
||||
- `detection.read_hints_enabled`:W06 读侧反序列化类型辅助(默认 true)
|
||||
- `detection.mq_read_hints_enabled`:MQ-R 读侧 Listener / parse 补强(默认 true)
|
||||
- `ignore.key_patterns`(锁 / 计数器 / token)
|
||||
- `ignore.mq_destinations`(MQ destination 忽略,默认真空)
|
||||
- `detection.min_confidence`、`max_field_depth`
|
||||
- `mode: notify`、`enabled: true`
|
||||
|
||||
@@ -97,6 +100,9 @@ ignore:
|
||||
# 忽略的写入方法(类全名#方法名)
|
||||
writer_methods: []
|
||||
|
||||
# 忽略的 MQ destination(topic 或 topic:tag,glob)
|
||||
mq_destinations: []
|
||||
|
||||
# 检测规则
|
||||
detection:
|
||||
# 启用的写入模式(默认已全部开启)
|
||||
@@ -107,9 +113,24 @@ detection:
|
||||
- W04 # redisTemplate 直写对象
|
||||
- W05 # opsForHash().put
|
||||
|
||||
# MQ 生产侧投递(Phase M1 + M2)
|
||||
mq_patterns:
|
||||
- MQ01 # rocketMQTemplate.syncSend
|
||||
- MQ02 # asyncSend / syncSendOrderly / sendOneWay
|
||||
- MQ03 # convertAndSend
|
||||
- MQ04 # MessageBuilder.withPayload / Message<T>
|
||||
- MQ05 # 先 toJSONString 再 send String
|
||||
- MQ-K01 # kafkaTemplate.send(topic, payload)
|
||||
- MQ-K02 # kafkaTemplate.send(topic, key, payload)
|
||||
- MQ-K03 # kafkaTemplate.send(ProducerRecord)
|
||||
- MQ-K04 # 先 JSON 序列化为 String 再 send
|
||||
|
||||
# W06:读侧反序列化类型辅助(不产生独立告警)
|
||||
read_hints_enabled: true
|
||||
|
||||
# MQ-R 读侧补强(Listener / parseObject)
|
||||
mq_read_hints_enabled: true
|
||||
|
||||
# 类型推断最低置信度,低于此值标记为低置信度提示
|
||||
min_confidence: 0.6
|
||||
|
||||
@@ -224,6 +245,7 @@ detection:
|
||||
- **不是写入模式**:不会单独因为「多了一处 parse」而告警
|
||||
- 能关联到 `redis get(key)` 时,还可补强 unresolved key
|
||||
- 覆盖优先级:`manual_mappings` > W06 > 写侧 AST
|
||||
- **仅补强 Redis 写入点**;MQ 投递点走 `mq_read_hints_enabled`(MQ-R)
|
||||
|
||||
关闭示例:
|
||||
|
||||
@@ -232,6 +254,18 @@ detection:
|
||||
read_hints_enabled: false
|
||||
```
|
||||
|
||||
### 3.5.2 detection.mq_patterns / ignore.mq_destinations(MQ)
|
||||
|
||||
| 配置项 | 说明 |
|
||||
|--------|------|
|
||||
| `mq_patterns` | RocketMQ:`MQ01`~`MQ05`;Kafka:`MQ-K01`~`MQ-K04`;默认已全部开启 |
|
||||
| `mq_read_hints_enabled` | MQ-R 读侧补强(Listener / parse),**默认 true** |
|
||||
| `ignore.mq_destinations` | 忽略 destination(glob),如 `benchmark:*` |
|
||||
|
||||
企微 MQ 块:`- Topic --> ...`,并带 `> **通道**: RocketMQ|Kafka`。
|
||||
|
||||
详见:[MQ序列化结构检测方案.md](MQ序列化结构检测实施方案.md)
|
||||
|
||||
### 3.6 manual_mappings
|
||||
|
||||
当自动推断不准确时使用。匹配优先级 **高于** 自动推断(按 `类全名#方法名` 覆盖 key 模式与 value 类型)。
|
||||
@@ -345,11 +379,3 @@ include_modules: [] # 全仓
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. 扩展:MQ 消息体检测(方案阶段)
|
||||
|
||||
MQ(**RocketMQ + Kafka**)消息体 Schema 变更监控方案已单独成文,**实现前不要求业务仓增配**。
|
||||
|
||||
详见:[MQ序列化结构检测方案.md](./MQ序列化结构检测方案.md)
|
||||
|
||||
届时预计新增:`detection.mq_patterns`(含 `MQ01~` 与 `MQ-K01~`)、`detection.mq_read_hints_enabled`、`ignore.mq_destinations`。
|
||||
@@ -3,6 +3,8 @@ package com.codechecker.cache.analyze;
|
||||
import com.codechecker.cache.config.CheckerConfig;
|
||||
import com.codechecker.cache.detector.CacheReadHint;
|
||||
import com.codechecker.cache.detector.CacheReadHintDetector;
|
||||
import com.codechecker.cache.detector.MqReadHintDetector;
|
||||
import com.codechecker.cache.detector.MqWritePointDetector;
|
||||
import com.codechecker.cache.detector.RedisWritePointDetector;
|
||||
import com.codechecker.cache.detector.WritePoint;
|
||||
import com.codechecker.cache.diff.ChangeType;
|
||||
@@ -92,13 +94,21 @@ public class SchemaCheckAnalyzer {
|
||||
}
|
||||
|
||||
Set<String> patterns = new HashSet<>(config.getDetection().getPatterns());
|
||||
Set<String> mqPatterns = new HashSet<>(config.getDetection().getMqPatterns());
|
||||
RedisWritePointDetector detectorNew = new RedisWritePointDetector(newIndex, patterns);
|
||||
RedisWritePointDetector detectorOld = new RedisWritePointDetector(oldIndex, patterns);
|
||||
MqWritePointDetector mqDetectorNew = mqPatterns.isEmpty()
|
||||
? null : new MqWritePointDetector(newIndex, mqPatterns);
|
||||
MqWritePointDetector mqDetectorOld = mqPatterns.isEmpty()
|
||||
? null : new MqWritePointDetector(oldIndex, mqPatterns);
|
||||
JavaSchemaExtractor extractorNew = new JavaSchemaExtractor(newIndex, config.getDetection().getMaxFieldDepth());
|
||||
JavaSchemaExtractor extractorOld = new JavaSchemaExtractor(oldIndex, config.getDetection().getMaxFieldDepth());
|
||||
SchemaDiffer differ = new SchemaDiffer();
|
||||
SkeletonJsonRenderer skeletonRenderer = new SkeletonJsonRenderer();
|
||||
|
||||
List<CacheReadHint> mqHintsNew = collectMqReadHints(newContents, newIndex);
|
||||
List<CacheReadHint> mqHintsOld = collectMqReadHints(oldContents, oldIndex);
|
||||
|
||||
List<SchemaChange> allChanges = new ArrayList<>();
|
||||
Map<String, KeyStructureChange> keyChanges = new LinkedHashMap<>();
|
||||
|
||||
@@ -110,13 +120,21 @@ public class SchemaCheckAnalyzer {
|
||||
boolean fileChanged = changedFiles.contains(path);
|
||||
String oldContent = fileChanged ? oldContents.get(path) : newContent;
|
||||
|
||||
List<WritePoint> newWps = detectorNew.detect(path, newContent);
|
||||
List<WritePoint> newWps = new ArrayList<>(detectorNew.detect(path, newContent));
|
||||
List<WritePoint> oldWps = oldContent == null
|
||||
? new ArrayList<>() : detectorOld.detect(path, oldContent);
|
||||
? new ArrayList<>() : new ArrayList<>(detectorOld.detect(path, oldContent));
|
||||
if (mqDetectorNew != null) {
|
||||
newWps.addAll(mqDetectorNew.detect(path, newContent));
|
||||
}
|
||||
if (mqDetectorOld != null && oldContent != null) {
|
||||
oldWps.addAll(mqDetectorOld.detect(path, oldContent));
|
||||
}
|
||||
applyReadHints(newWps, path, newContent, newIndex);
|
||||
if (oldContent != null) {
|
||||
applyReadHints(oldWps, path, oldContent, oldIndex);
|
||||
}
|
||||
applyMqReadHints(newWps, mqHintsNew);
|
||||
applyMqReadHints(oldWps, mqHintsOld);
|
||||
newWps.forEach(this::applyManualMappings);
|
||||
oldWps.forEach(this::applyManualMappings);
|
||||
|
||||
@@ -128,7 +146,7 @@ public class SchemaCheckAnalyzer {
|
||||
|
||||
for (WritePoint nw : newWps) {
|
||||
newSigs.add(nw.signature());
|
||||
if (isKeyIgnored(nw.getResolvedKeyPattern()) || isWriterIgnored(nw)) {
|
||||
if (isWritePointIgnored(nw) || isWriterIgnored(nw)) {
|
||||
continue;
|
||||
}
|
||||
WritePoint ow = oldBySig.get(nw.signature());
|
||||
@@ -151,7 +169,9 @@ public class SchemaCheckAnalyzer {
|
||||
} else if (fileChanged) {
|
||||
SchemaChange c = new SchemaChange(ChangeType.WRITE_POINT_ADDED);
|
||||
fillFromWritePoint(c, nw);
|
||||
c.setMessage("新增缓存写入点,value 类型: " + displayType(nw));
|
||||
c.setMessage(nw.isMq()
|
||||
? "新增 MQ 投递点,value 类型: " + displayType(nw)
|
||||
: "新增缓存写入点,value 类型: " + displayType(nw));
|
||||
allChanges.add(c);
|
||||
TypeSchema newSchema = extractorNew.extract(nw.getResolvedValueType(), nw.isRootArray());
|
||||
mergeKeyChange(keyChanges, nw,
|
||||
@@ -166,10 +186,12 @@ public class SchemaCheckAnalyzer {
|
||||
if (fileChanged) {
|
||||
for (WritePoint ow : oldWps) {
|
||||
if (!newSigs.contains(ow.signature())
|
||||
&& !isKeyIgnored(ow.getResolvedKeyPattern()) && !isWriterIgnored(ow)) {
|
||||
&& !isWritePointIgnored(ow) && !isWriterIgnored(ow)) {
|
||||
SchemaChange c = new SchemaChange(ChangeType.WRITE_POINT_REMOVED);
|
||||
fillFromWritePoint(c, ow);
|
||||
c.setMessage("删除缓存写入点,原 value 类型: " + displayType(ow));
|
||||
c.setMessage(ow.isMq()
|
||||
? "删除 MQ 投递点,原 value 类型: " + displayType(ow)
|
||||
: "删除缓存写入点,原 value 类型: " + displayType(ow));
|
||||
allChanges.add(c);
|
||||
TypeSchema oldSchema = extractorOld.extract(ow.getResolvedValueType(), ow.isRootArray());
|
||||
mergeKeyChange(keyChanges, ow,
|
||||
@@ -198,8 +220,12 @@ public class SchemaCheckAnalyzer {
|
||||
n.setWriteLocation(wp.location());
|
||||
n.setValueType(displayType(wp));
|
||||
n.setKeyUnresolved(isUnresolvedKey(wp.getResolvedKeyPattern()));
|
||||
n.setChannel(wp.getChannel());
|
||||
return n;
|
||||
});
|
||||
if (kc.getChannel() == null || kc.getChannel().isEmpty()) {
|
||||
kc.setChannel(wp.getChannel());
|
||||
}
|
||||
if (kc.getWriteLocation() == null || kc.getWriteLocation().isEmpty()) {
|
||||
kc.setWriteLocation(wp.location());
|
||||
}
|
||||
@@ -229,11 +255,12 @@ public class SchemaCheckAnalyzer {
|
||||
|
||||
/** 已解析 key 按模式聚合;未解析按「位置+表达式」拆分,避免串单。 */
|
||||
private String aggregationKey(WritePoint wp) {
|
||||
String channel = wp.getChannel() == null ? WritePoint.CHANNEL_REDIS : wp.getChannel();
|
||||
String pattern = wp.getResolvedKeyPattern();
|
||||
if (!isUnresolvedKey(pattern)) {
|
||||
return pattern == null ? "<unknown>" : pattern;
|
||||
return channel + "|" + (pattern == null ? "<unknown>" : pattern);
|
||||
}
|
||||
return "unknown|" + nvl(wp.location()) + "|" + nvl(wp.getKeyExpression());
|
||||
return channel + "|unknown|" + nvl(wp.location()) + "|" + nvl(wp.getKeyExpression());
|
||||
}
|
||||
|
||||
private boolean isUnresolvedKey(String keyPattern) {
|
||||
@@ -274,10 +301,12 @@ public class SchemaCheckAnalyzer {
|
||||
paths.add(c.getOldValue());
|
||||
}
|
||||
if (c.getChangeType() == ChangeType.WRAPPER_ADDED
|
||||
&& c.getFieldPath() != null) {
|
||||
|| c.getChangeType() == ChangeType.WRAPPER_REMOVED) {
|
||||
if (c.getFieldPath() != null) {
|
||||
paths.add(c.getFieldPath());
|
||||
}
|
||||
}
|
||||
}
|
||||
return paths;
|
||||
}
|
||||
|
||||
@@ -331,8 +360,14 @@ public class SchemaCheckAnalyzer {
|
||||
}
|
||||
|
||||
private String changeDedupKey(SchemaChange c) {
|
||||
return c.getChangeType() + "|" + c.getKeyPattern() + "|"
|
||||
+ c.getWriteLocation() + "|" + c.getFieldPath();
|
||||
ChangeType t = c.getChangeType();
|
||||
// 新增/删除投递点:按位置区分
|
||||
if (t == ChangeType.WRITE_POINT_ADDED || t == ChangeType.WRITE_POINT_REMOVED) {
|
||||
return t + "|" + c.getKeyPattern() + "|" + c.getWriteLocation();
|
||||
}
|
||||
// 同一 destination 上多处 send 同源 VO:字段级结构变更只保留一条
|
||||
return t + "|" + c.getKeyPattern() + "|" + c.getFieldPath()
|
||||
+ "|" + nvl(c.getOldValue()) + "|" + nvl(c.getNewValue());
|
||||
}
|
||||
|
||||
private void enrich(List<SchemaChange> changes, WritePoint wp, double confidence) {
|
||||
@@ -356,9 +391,7 @@ public class SchemaCheckAnalyzer {
|
||||
if (isSuppressed(c)) {
|
||||
continue;
|
||||
}
|
||||
String dedupKey = c.getChangeType() + "|" + c.getKeyPattern() + "|"
|
||||
+ c.getWriteLocation() + "|" + c.getFieldPath();
|
||||
if (seen.add(dedupKey)) {
|
||||
if (seen.add(changeDedupKey(c))) {
|
||||
result.add(c);
|
||||
}
|
||||
}
|
||||
@@ -446,10 +479,39 @@ public class SchemaCheckAnalyzer {
|
||||
return;
|
||||
}
|
||||
for (WritePoint wp : writePoints) {
|
||||
if (wp.isMq()) {
|
||||
continue; // MQ 投递点由 MQ-R 补强
|
||||
}
|
||||
enrichWritePointFromHints(wp, hints);
|
||||
}
|
||||
}
|
||||
|
||||
private List<CacheReadHint> collectMqReadHints(Map<String, String> contents, SourceIndex index) {
|
||||
if (!config.getDetection().isMqReadHintsEnabled() || contents == null || contents.isEmpty()) {
|
||||
return Collections.emptyList();
|
||||
}
|
||||
MqReadHintDetector detector = new MqReadHintDetector(index);
|
||||
List<CacheReadHint> all = new ArrayList<>();
|
||||
for (Map.Entry<String, String> e : contents.entrySet()) {
|
||||
all.addAll(detector.detect(e.getKey(), e.getValue()));
|
||||
}
|
||||
return all;
|
||||
}
|
||||
|
||||
/** MQ-R:按 destination 匹配,补强低置信度 / 缺类型的 MQ 投递点。 */
|
||||
private void applyMqReadHints(List<WritePoint> writePoints, List<CacheReadHint> hints) {
|
||||
if (!config.getDetection().isMqReadHintsEnabled()
|
||||
|| writePoints == null || writePoints.isEmpty()
|
||||
|| hints == null || hints.isEmpty()) {
|
||||
return;
|
||||
}
|
||||
for (WritePoint wp : writePoints) {
|
||||
if (wp.isMq()) {
|
||||
enrichWritePointFromHints(wp, hints);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void enrichWritePointFromHints(WritePoint wp, List<CacheReadHint> hints) {
|
||||
boolean needType = wp.getResolvedValueType() == null || wp.getResolvedValueType().isEmpty()
|
||||
|| wp.getConfidence() < config.getDetection().getMinConfidence();
|
||||
@@ -553,6 +615,13 @@ public class SchemaCheckAnalyzer {
|
||||
return false;
|
||||
}
|
||||
|
||||
private boolean isWritePointIgnored(WritePoint wp) {
|
||||
if (wp.isMq()) {
|
||||
return isMqDestinationIgnored(wp.getResolvedKeyPattern());
|
||||
}
|
||||
return isKeyIgnored(wp.getResolvedKeyPattern());
|
||||
}
|
||||
|
||||
private boolean isKeyIgnored(String keyPattern) {
|
||||
if (keyPattern == null) {
|
||||
return false;
|
||||
@@ -565,6 +634,18 @@ public class SchemaCheckAnalyzer {
|
||||
return false;
|
||||
}
|
||||
|
||||
private boolean isMqDestinationIgnored(String destination) {
|
||||
if (destination == null) {
|
||||
return false;
|
||||
}
|
||||
for (String glob : config.getIgnore().getMqDestinations()) {
|
||||
if (GlobMatcher.matches(glob, destination)) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private boolean isWriterIgnored(WritePoint wp) {
|
||||
String sig = wp.getEnclosingClass() + "#" + wp.getEnclosingMethod();
|
||||
return config.getIgnore().getWriterMethods().contains(sig);
|
||||
|
||||
@@ -80,6 +80,8 @@ public class CheckerConfig {
|
||||
private List<String> keyPatterns = new ArrayList<>();
|
||||
private List<String> filePatterns = new ArrayList<>();
|
||||
private List<String> writerMethods = new ArrayList<>();
|
||||
/** 忽略的 MQ destination 模式(topic / topic:tag) */
|
||||
private List<String> mqDestinations = new ArrayList<>();
|
||||
|
||||
public List<String> getKeyPatterns() {
|
||||
return keyPatterns;
|
||||
@@ -104,14 +106,26 @@ public class CheckerConfig {
|
||||
public void setWriterMethods(List<String> writerMethods) {
|
||||
this.writerMethods = writerMethods;
|
||||
}
|
||||
|
||||
public List<String> getMqDestinations() {
|
||||
return mqDestinations;
|
||||
}
|
||||
|
||||
public void setMqDestinations(List<String> mqDestinations) {
|
||||
this.mqDestinations = mqDestinations;
|
||||
}
|
||||
}
|
||||
|
||||
public static class Detection {
|
||||
private List<String> patterns = new ArrayList<>();
|
||||
/** MQ 投递检测模式:MQ01~MQ05、MQ-K01~MQ-K04 */
|
||||
private List<String> mqPatterns = new ArrayList<>();
|
||||
private double minConfidence = 0.6;
|
||||
private int maxFieldDepth = 8;
|
||||
/** W06:是否启用读侧反序列化类型辅助补强 */
|
||||
private boolean readHintsEnabled = true;
|
||||
/** MQ-R:读侧 Listener / parse 补强 */
|
||||
private boolean mqReadHintsEnabled = true;
|
||||
|
||||
public List<String> getPatterns() {
|
||||
return patterns;
|
||||
@@ -121,6 +135,14 @@ public class CheckerConfig {
|
||||
this.patterns = patterns;
|
||||
}
|
||||
|
||||
public List<String> getMqPatterns() {
|
||||
return mqPatterns;
|
||||
}
|
||||
|
||||
public void setMqPatterns(List<String> mqPatterns) {
|
||||
this.mqPatterns = mqPatterns;
|
||||
}
|
||||
|
||||
public double getMinConfidence() {
|
||||
return minConfidence;
|
||||
}
|
||||
@@ -144,6 +166,14 @@ public class CheckerConfig {
|
||||
public void setReadHintsEnabled(boolean readHintsEnabled) {
|
||||
this.readHintsEnabled = readHintsEnabled;
|
||||
}
|
||||
|
||||
public boolean isMqReadHintsEnabled() {
|
||||
return mqReadHintsEnabled;
|
||||
}
|
||||
|
||||
public void setMqReadHintsEnabled(boolean mqReadHintsEnabled) {
|
||||
this.mqReadHintsEnabled = mqReadHintsEnabled;
|
||||
}
|
||||
}
|
||||
|
||||
public static class ManualMapping {
|
||||
|
||||
@@ -94,13 +94,16 @@ public final class ConfigLoader {
|
||||
ig.setKeyPatterns(strList(ignore.get("key_patterns")));
|
||||
ig.setFilePatterns(strList(ignore.get("file_patterns")));
|
||||
ig.setWriterMethods(strList(ignore.get("writer_methods")));
|
||||
ig.setMqDestinations(strList(ignore.get("mq_destinations")));
|
||||
|
||||
Map<String, Object> detection = asMap(map.get("detection"));
|
||||
CheckerConfig.Detection d = config.getDetection();
|
||||
d.setPatterns(strList(detection.get("patterns")));
|
||||
d.setMqPatterns(strList(detection.get("mq_patterns")));
|
||||
d.setMinConfidence(dbl(detection, "min_confidence", 0.6));
|
||||
d.setMaxFieldDepth((int) lng(detection, "max_field_depth", 8));
|
||||
d.setReadHintsEnabled(bool(detection, "read_hints_enabled", true));
|
||||
d.setMqReadHintsEnabled(bool(detection, "mq_read_hints_enabled", true));
|
||||
|
||||
Map<String, Object> severityOverrides = asMap(map.get("severity_overrides"));
|
||||
Map<String, String> so = new LinkedHashMap<>();
|
||||
|
||||
365
src/main/java/com/codechecker/cache/detector/MqReadHintDetector.java
vendored
Normal file
365
src/main/java/com/codechecker/cache/detector/MqReadHintDetector.java
vendored
Normal file
@@ -0,0 +1,365 @@
|
||||
package com.codechecker.cache.detector;
|
||||
|
||||
import com.codechecker.cache.key.RedisKeyResolver;
|
||||
import com.codechecker.cache.schema.SourceIndex;
|
||||
import com.github.javaparser.StaticJavaParser;
|
||||
import com.github.javaparser.ast.CompilationUnit;
|
||||
import com.github.javaparser.ast.body.ClassOrInterfaceDeclaration;
|
||||
import com.github.javaparser.ast.body.MethodDeclaration;
|
||||
import com.github.javaparser.ast.body.Parameter;
|
||||
import com.github.javaparser.ast.expr.AnnotationExpr;
|
||||
import com.github.javaparser.ast.expr.ArrayInitializerExpr;
|
||||
import com.github.javaparser.ast.expr.ClassExpr;
|
||||
import com.github.javaparser.ast.expr.Expression;
|
||||
import com.github.javaparser.ast.expr.MarkerAnnotationExpr;
|
||||
import com.github.javaparser.ast.expr.MemberValuePair;
|
||||
import com.github.javaparser.ast.expr.MethodCallExpr;
|
||||
import com.github.javaparser.ast.expr.NormalAnnotationExpr;
|
||||
import com.github.javaparser.ast.expr.SingleMemberAnnotationExpr;
|
||||
import com.github.javaparser.ast.type.ClassOrInterfaceType;
|
||||
import com.github.javaparser.ast.type.Type;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.Arrays;
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
import java.util.Optional;
|
||||
import java.util.Set;
|
||||
|
||||
/**
|
||||
* MQ-R:消费侧类型提示,补强同 destination 生产点的 value 类型(不单独告警)。
|
||||
* <ul>
|
||||
* <li>MQ-R01:{@code RocketMQListener<T>} + {@code @RocketMQMessageListener}</li>
|
||||
* <li>MQ-R02:{@code @KafkaListener} 非 String 参数类型</li>
|
||||
* <li>MQ-R03:Listener 内 {@code parseObject/parseArray(..., Xxx.class)}</li>
|
||||
* </ul>
|
||||
*/
|
||||
public class MqReadHintDetector {
|
||||
|
||||
private static final Set<String> OBJECT_PARSE = new HashSet<>(Arrays.asList(
|
||||
"parseObject", "parse", "getJsonToBean", "toJavaObject", "readValue"));
|
||||
private static final Set<String> ARRAY_PARSE = new HashSet<>(Arrays.asList(
|
||||
"parseArray", "getJsonToList", "parseArrayObject"));
|
||||
private static final Set<String> SKIP_PARAM_TYPES = new HashSet<>(Arrays.asList(
|
||||
"String", "byte", "Byte", "ConsumerRecord", "MessageExt", "Message"));
|
||||
|
||||
private final SourceIndex index;
|
||||
private final RedisKeyResolver keyResolver;
|
||||
|
||||
public MqReadHintDetector(SourceIndex index) {
|
||||
this.index = index;
|
||||
this.keyResolver = new RedisKeyResolver(index);
|
||||
}
|
||||
|
||||
public List<CacheReadHint> detect(String filePath, String content) {
|
||||
List<CacheReadHint> result = new ArrayList<>();
|
||||
if (content == null || content.isEmpty()) {
|
||||
return result;
|
||||
}
|
||||
CompilationUnit cu;
|
||||
try {
|
||||
cu = StaticJavaParser.parse(content);
|
||||
} catch (RuntimeException e) {
|
||||
return result;
|
||||
}
|
||||
for (ClassOrInterfaceDeclaration clazz : cu.findAll(ClassOrInterfaceDeclaration.class)) {
|
||||
result.addAll(detectRocketListener(clazz, filePath));
|
||||
for (MethodDeclaration md : clazz.getMethods()) {
|
||||
result.addAll(detectKafkaListener(md, clazz, filePath));
|
||||
result.addAll(detectParseInListener(md, clazz, filePath));
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/** MQ-R01 */
|
||||
private List<CacheReadHint> detectRocketListener(ClassOrInterfaceDeclaration clazz, String filePath) {
|
||||
List<CacheReadHint> result = new ArrayList<>();
|
||||
Optional<ClassOrInterfaceType> listenerType = findRocketListenerType(clazz);
|
||||
if (!listenerType.isPresent()) {
|
||||
return result;
|
||||
}
|
||||
AnnotationExpr ann = findAnnotation(clazz.getAnnotations(), "RocketMQMessageListener");
|
||||
if (ann == null) {
|
||||
return result;
|
||||
}
|
||||
String dest = resolveRocketDestination(ann, clazz);
|
||||
InferredType payload = resolveTypeArg(listenerType.get(), index.get(
|
||||
clazz.getFullyQualifiedName().orElse(clazz.getNameAsString())));
|
||||
if (payload.fqn == null) {
|
||||
return result;
|
||||
}
|
||||
CacheReadHint hint = baseHint(filePath, clazz, "<listener>", dest, payload);
|
||||
hint.setConfidence(0.85);
|
||||
result.add(hint);
|
||||
return result;
|
||||
}
|
||||
|
||||
/** MQ-R02 */
|
||||
private List<CacheReadHint> detectKafkaListener(MethodDeclaration md,
|
||||
ClassOrInterfaceDeclaration clazz,
|
||||
String filePath) {
|
||||
List<CacheReadHint> result = new ArrayList<>();
|
||||
AnnotationExpr ann = findAnnotation(md.getAnnotations(), "KafkaListener");
|
||||
if (ann == null) {
|
||||
return result;
|
||||
}
|
||||
List<String> topics = resolveKafkaTopics(ann, clazz);
|
||||
if (topics.isEmpty()) {
|
||||
return result;
|
||||
}
|
||||
SourceIndex.IndexedType context = index.get(
|
||||
clazz.getFullyQualifiedName().orElse(clazz.getNameAsString()));
|
||||
InferredType payload = null;
|
||||
for (Parameter p : md.getParameters()) {
|
||||
InferredType t = resolveParamPayload(p.getType(), context);
|
||||
if (t != null && t.fqn != null) {
|
||||
payload = t;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (payload == null) {
|
||||
return result;
|
||||
}
|
||||
for (String topic : topics) {
|
||||
CacheReadHint hint = baseHint(filePath, clazz, md.getNameAsString(), topic, payload);
|
||||
hint.setConfidence(0.85);
|
||||
result.add(hint);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/** MQ-R03:在已标注 KafkaListener 的方法,或 RocketMQListener 类内 parse */
|
||||
private List<CacheReadHint> detectParseInListener(MethodDeclaration md,
|
||||
ClassOrInterfaceDeclaration clazz,
|
||||
String filePath) {
|
||||
List<CacheReadHint> result = new ArrayList<>();
|
||||
boolean kafka = findAnnotation(md.getAnnotations(), "KafkaListener") != null;
|
||||
boolean rocket = findRocketListenerType(clazz).isPresent();
|
||||
if (!kafka && !rocket) {
|
||||
return result;
|
||||
}
|
||||
List<String> destinations = new ArrayList<>();
|
||||
if (kafka) {
|
||||
destinations.addAll(resolveKafkaTopics(
|
||||
findAnnotation(md.getAnnotations(), "KafkaListener"), clazz));
|
||||
}
|
||||
if (rocket) {
|
||||
AnnotationExpr ann = findAnnotation(clazz.getAnnotations(), "RocketMQMessageListener");
|
||||
if (ann != null) {
|
||||
String dest = resolveRocketDestination(ann, clazz);
|
||||
if (dest != null && !dest.isEmpty()) {
|
||||
destinations.add(dest);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (destinations.isEmpty()) {
|
||||
destinations.add(null);
|
||||
}
|
||||
|
||||
SourceIndex.IndexedType context = index.get(
|
||||
clazz.getFullyQualifiedName().orElse(clazz.getNameAsString()));
|
||||
for (MethodCallExpr mce : md.findAll(MethodCallExpr.class)) {
|
||||
String name = mce.getNameAsString();
|
||||
boolean array = ARRAY_PARSE.contains(name);
|
||||
boolean object = OBJECT_PARSE.contains(name);
|
||||
if (!array && !object || mce.getArguments().size() < 2) {
|
||||
continue;
|
||||
}
|
||||
Expression classArg = mce.getArgument(1);
|
||||
if (!(classArg instanceof ClassExpr)) {
|
||||
continue;
|
||||
}
|
||||
Type type = ((ClassExpr) classArg).getType();
|
||||
if (!(type instanceof ClassOrInterfaceType)) {
|
||||
continue;
|
||||
}
|
||||
String fqn = index.resolveFqn(((ClassOrInterfaceType) type).getNameWithScope(), context);
|
||||
if (fqn == null) {
|
||||
fqn = index.resolveFqn(((ClassOrInterfaceType) type).getNameAsString(), context);
|
||||
}
|
||||
if (fqn == null) {
|
||||
continue;
|
||||
}
|
||||
InferredType payload = new InferredType(fqn, array);
|
||||
for (String dest : destinations) {
|
||||
CacheReadHint hint = baseHint(filePath, clazz, md.getNameAsString(), dest, payload);
|
||||
hint.setLineNumber(mce.getBegin().map(p -> p.line).orElse(0));
|
||||
hint.setConfidence(dest == null ? 0.7 : 0.8);
|
||||
result.add(hint);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private CacheReadHint baseHint(String filePath, ClassOrInterfaceDeclaration clazz,
|
||||
String method, String dest, InferredType payload) {
|
||||
CacheReadHint hint = new CacheReadHint();
|
||||
hint.setFilePath(filePath);
|
||||
hint.setLineNumber(clazz.getBegin().map(p -> p.line).orElse(0));
|
||||
hint.setEnclosingClass(clazz.getFullyQualifiedName().orElse(clazz.getNameAsString()));
|
||||
hint.setEnclosingMethod(method);
|
||||
hint.setResolvedKeyPattern(dest);
|
||||
hint.setResolvedValueType(payload.fqn);
|
||||
hint.setRootArray(payload.isArray);
|
||||
return hint;
|
||||
}
|
||||
|
||||
private Optional<ClassOrInterfaceType> findRocketListenerType(ClassOrInterfaceDeclaration clazz) {
|
||||
for (ClassOrInterfaceType t : clazz.getImplementedTypes()) {
|
||||
String n = t.getNameAsString();
|
||||
if ("RocketMQListener".equals(n) || "RocketMQReplyListener".equals(n)) {
|
||||
return Optional.of(t);
|
||||
}
|
||||
}
|
||||
return Optional.empty();
|
||||
}
|
||||
|
||||
private InferredType resolveParamPayload(Type type, SourceIndex.IndexedType context) {
|
||||
if (!(type instanceof ClassOrInterfaceType)) {
|
||||
return null;
|
||||
}
|
||||
ClassOrInterfaceType cit = (ClassOrInterfaceType) type;
|
||||
String simple = cit.getNameAsString();
|
||||
if (SKIP_PARAM_TYPES.contains(simple) && !"ConsumerRecord".equals(simple)) {
|
||||
return null;
|
||||
}
|
||||
if ("ConsumerRecord".equals(simple) || "List".equals(simple) || "ArrayList".equals(simple)) {
|
||||
Optional<Type> last = cit.getTypeArguments()
|
||||
.filter(a -> !a.isEmpty())
|
||||
.map(a -> a.get(a.size() - 1));
|
||||
if (!last.isPresent()) {
|
||||
return null;
|
||||
}
|
||||
InferredType inner = resolveTypeArgFromType(last.get(), context);
|
||||
if (inner != null && ("List".equals(simple) || "ArrayList".equals(simple))) {
|
||||
return new InferredType(inner.fqn, true);
|
||||
}
|
||||
return inner;
|
||||
}
|
||||
return resolveTypeArgFromType(type, context);
|
||||
}
|
||||
|
||||
private InferredType resolveTypeArg(ClassOrInterfaceType listenerType, SourceIndex.IndexedType context) {
|
||||
Optional<Type> arg = listenerType.getTypeArguments().filter(a -> !a.isEmpty()).map(a -> a.get(0));
|
||||
if (!arg.isPresent()) {
|
||||
return new InferredType(null, false);
|
||||
}
|
||||
return resolveTypeArgFromType(arg.get(), context);
|
||||
}
|
||||
|
||||
private InferredType resolveTypeArgFromType(Type type, SourceIndex.IndexedType context) {
|
||||
if (!(type instanceof ClassOrInterfaceType)) {
|
||||
return new InferredType(null, false);
|
||||
}
|
||||
ClassOrInterfaceType cit = (ClassOrInterfaceType) type;
|
||||
if ("List".equals(cit.getNameAsString()) || "ArrayList".equals(cit.getNameAsString())) {
|
||||
Optional<Type> el = cit.getTypeArguments().filter(a -> !a.isEmpty()).map(a -> a.get(0));
|
||||
if (el.isPresent() && el.get() instanceof ClassOrInterfaceType) {
|
||||
return new InferredType(resolveFqn((ClassOrInterfaceType) el.get(), context), true);
|
||||
}
|
||||
return new InferredType(null, true);
|
||||
}
|
||||
String simple = cit.getNameAsString();
|
||||
if (SKIP_PARAM_TYPES.contains(simple)) {
|
||||
return new InferredType(null, false);
|
||||
}
|
||||
return new InferredType(resolveFqn(cit, context), false);
|
||||
}
|
||||
|
||||
private String resolveFqn(ClassOrInterfaceType cit, SourceIndex.IndexedType context) {
|
||||
String fqn = index.resolveFqn(cit.getNameWithScope(), context);
|
||||
if (fqn == null) {
|
||||
fqn = index.resolveFqn(cit.getNameAsString(), context);
|
||||
}
|
||||
return fqn;
|
||||
}
|
||||
|
||||
private String resolveRocketDestination(AnnotationExpr ann, ClassOrInterfaceDeclaration clazz) {
|
||||
SourceIndex.IndexedType context = index.get(
|
||||
clazz.getFullyQualifiedName().orElse(clazz.getNameAsString()));
|
||||
Expression topicExpr = annotationValue(ann, "topic");
|
||||
if (topicExpr == null) {
|
||||
return null;
|
||||
}
|
||||
String topicPattern = keyResolver.resolve(topicExpr, clazz, context);
|
||||
Expression tagExpr = annotationValue(ann, "selectorExpression");
|
||||
if (tagExpr != null) {
|
||||
String tagPattern = keyResolver.resolve(tagExpr, clazz, context);
|
||||
if (tagPattern != null && !tagPattern.isEmpty()
|
||||
&& !"*".equals(tagPattern) && !tagPattern.contains("||")) {
|
||||
return topicPattern + ":" + tagPattern;
|
||||
}
|
||||
}
|
||||
return topicPattern;
|
||||
}
|
||||
|
||||
private List<String> resolveKafkaTopics(AnnotationExpr ann, ClassOrInterfaceDeclaration clazz) {
|
||||
List<String> result = new ArrayList<>();
|
||||
if (ann == null) {
|
||||
return result;
|
||||
}
|
||||
SourceIndex.IndexedType context = index.get(
|
||||
clazz.getFullyQualifiedName().orElse(clazz.getNameAsString()));
|
||||
Expression topicsExpr = annotationValue(ann, "topics");
|
||||
if (topicsExpr == null && ann instanceof SingleMemberAnnotationExpr) {
|
||||
topicsExpr = ((SingleMemberAnnotationExpr) ann).getMemberValue();
|
||||
}
|
||||
if (topicsExpr == null) {
|
||||
return result;
|
||||
}
|
||||
List<Expression> items = new ArrayList<>();
|
||||
if (topicsExpr instanceof ArrayInitializerExpr) {
|
||||
items.addAll(((ArrayInitializerExpr) topicsExpr).getValues());
|
||||
} else {
|
||||
items.add(topicsExpr);
|
||||
}
|
||||
for (Expression item : items) {
|
||||
String resolved = keyResolver.resolve(item, clazz, context);
|
||||
if (resolved != null && !resolved.isEmpty()) {
|
||||
result.add(resolved);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private static Expression annotationValue(AnnotationExpr ann, String name) {
|
||||
if (ann instanceof NormalAnnotationExpr) {
|
||||
for (MemberValuePair pair : ((NormalAnnotationExpr) ann).getPairs()) {
|
||||
if (name.equals(pair.getNameAsString())) {
|
||||
return pair.getValue();
|
||||
}
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private static AnnotationExpr findAnnotation(List<AnnotationExpr> annotations, String simpleName) {
|
||||
for (AnnotationExpr ann : annotations) {
|
||||
String n;
|
||||
if (ann instanceof MarkerAnnotationExpr) {
|
||||
n = ((MarkerAnnotationExpr) ann).getNameAsString();
|
||||
} else if (ann instanceof SingleMemberAnnotationExpr) {
|
||||
n = ((SingleMemberAnnotationExpr) ann).getNameAsString();
|
||||
} else if (ann instanceof NormalAnnotationExpr) {
|
||||
n = ((NormalAnnotationExpr) ann).getNameAsString();
|
||||
} else {
|
||||
n = ann.getNameAsString();
|
||||
}
|
||||
if (n.equals(simpleName) || n.endsWith("." + simpleName)) {
|
||||
return ann;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private static final class InferredType {
|
||||
final String fqn;
|
||||
final boolean isArray;
|
||||
|
||||
InferredType(String fqn, boolean isArray) {
|
||||
this.fqn = fqn;
|
||||
this.isArray = isArray;
|
||||
}
|
||||
}
|
||||
}
|
||||
548
src/main/java/com/codechecker/cache/detector/MqWritePointDetector.java
vendored
Normal file
548
src/main/java/com/codechecker/cache/detector/MqWritePointDetector.java
vendored
Normal file
@@ -0,0 +1,548 @@
|
||||
package com.codechecker.cache.detector;
|
||||
|
||||
import com.codechecker.cache.key.RedisKeyResolver;
|
||||
import com.codechecker.cache.schema.SourceIndex;
|
||||
import com.github.javaparser.StaticJavaParser;
|
||||
import com.github.javaparser.ast.CompilationUnit;
|
||||
import com.github.javaparser.ast.body.CallableDeclaration;
|
||||
import com.github.javaparser.ast.body.ClassOrInterfaceDeclaration;
|
||||
import com.github.javaparser.ast.body.FieldDeclaration;
|
||||
import com.github.javaparser.ast.body.MethodDeclaration;
|
||||
import com.github.javaparser.ast.body.Parameter;
|
||||
import com.github.javaparser.ast.body.VariableDeclarator;
|
||||
import com.github.javaparser.ast.expr.BinaryExpr;
|
||||
import com.github.javaparser.ast.expr.BooleanLiteralExpr;
|
||||
import com.github.javaparser.ast.expr.CastExpr;
|
||||
import com.github.javaparser.ast.expr.Expression;
|
||||
import com.github.javaparser.ast.expr.IntegerLiteralExpr;
|
||||
import com.github.javaparser.ast.expr.LongLiteralExpr;
|
||||
import com.github.javaparser.ast.expr.MethodCallExpr;
|
||||
import com.github.javaparser.ast.expr.NameExpr;
|
||||
import com.github.javaparser.ast.expr.NullLiteralExpr;
|
||||
import com.github.javaparser.ast.expr.ObjectCreationExpr;
|
||||
import com.github.javaparser.ast.expr.StringLiteralExpr;
|
||||
import com.github.javaparser.ast.type.ClassOrInterfaceType;
|
||||
import com.github.javaparser.ast.type.Type;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.Arrays;
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
import java.util.Optional;
|
||||
import java.util.Set;
|
||||
|
||||
/**
|
||||
* 检测 RocketMQ / Kafka 生产侧投递点(Phase M1 + M2)。
|
||||
* <ul>
|
||||
* <li>M1:MQ01/MQ02、MQ-K01/MQ-K02</li>
|
||||
* <li>M2:MQ03 convertAndSend、MQ04 MessageBuilder/`Message<T>`、MQ05 JSON 字符串、
|
||||
* MQ-K03 ProducerRecord、MQ-K04 JSON 字符串</li>
|
||||
* </ul>
|
||||
*/
|
||||
public class MqWritePointDetector {
|
||||
|
||||
private static final Set<String> SERIALIZE_METHODS = new HashSet<>(Arrays.asList(
|
||||
"toJSONString", "toJsonString", "getObjectToString", "toJsonStr", "writeValueAsString"));
|
||||
private static final Set<String> ROCKET_SYNC = new HashSet<>(Arrays.asList("syncSend"));
|
||||
private static final Set<String> ROCKET_ASYNC = new HashSet<>(Arrays.asList(
|
||||
"asyncSend", "syncSendOrderly", "sendOneWay", "asyncSendOrderly"));
|
||||
private static final Set<String> ROCKET_CONVERT = new HashSet<>(Arrays.asList("convertAndSend"));
|
||||
private static final Set<String> TRIVIAL_VALUE_CALLS = new HashSet<>(Arrays.asList(
|
||||
"randomUUID", "toString", "valueOf"));
|
||||
private static final Set<String> COLLECTION_SIMPLE = new HashSet<>(Arrays.asList(
|
||||
"List", "ArrayList", "LinkedList", "Set", "HashSet", "Collection"));
|
||||
private static final Set<String> ENVELOPE_TYPES = new HashSet<>(Arrays.asList(
|
||||
"Message", "MessageExt", "ProducerRecord"));
|
||||
private static final Set<String> IGNORE_BARE_TYPES = new HashSet<>(Arrays.asList(
|
||||
"String", "byte", "Byte"));
|
||||
|
||||
private final SourceIndex index;
|
||||
private final Set<String> enabledPatterns;
|
||||
private final RedisKeyResolver keyResolver;
|
||||
|
||||
public MqWritePointDetector(SourceIndex index, Set<String> enabledPatterns) {
|
||||
this.index = index;
|
||||
this.enabledPatterns = enabledPatterns == null ? new HashSet<>() : enabledPatterns;
|
||||
this.keyResolver = new RedisKeyResolver(index);
|
||||
}
|
||||
|
||||
public List<WritePoint> detect(String filePath, String content) {
|
||||
List<WritePoint> result = new ArrayList<>();
|
||||
if (content == null || content.isEmpty() || enabledPatterns.isEmpty()) {
|
||||
return result;
|
||||
}
|
||||
CompilationUnit cu;
|
||||
try {
|
||||
cu = StaticJavaParser.parse(content);
|
||||
} catch (RuntimeException e) {
|
||||
return result;
|
||||
}
|
||||
for (MethodCallExpr mce : cu.findAll(MethodCallExpr.class)) {
|
||||
WritePoint wp = tryDetectRocketMq(mce, filePath);
|
||||
if (wp == null) {
|
||||
wp = tryDetectKafka(mce, filePath);
|
||||
}
|
||||
if (wp != null) {
|
||||
result.add(wp);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private WritePoint tryDetectRocketMq(MethodCallExpr mce, String filePath) {
|
||||
String method = mce.getNameAsString();
|
||||
String basePattern = null;
|
||||
if (ROCKET_SYNC.contains(method) && enabledPatterns.contains("MQ01")) {
|
||||
basePattern = "MQ01";
|
||||
} else if (ROCKET_ASYNC.contains(method) && enabledPatterns.contains("MQ02")) {
|
||||
basePattern = "MQ02";
|
||||
} else if (ROCKET_CONVERT.contains(method) && enabledPatterns.contains("MQ03")) {
|
||||
basePattern = "MQ03";
|
||||
}
|
||||
if (basePattern == null || !isRocketMqScope(mce) || mce.getArguments().size() < 2) {
|
||||
return null;
|
||||
}
|
||||
return buildMqWritePoint(mce, filePath, basePattern, WritePoint.CHANNEL_ROCKETMQ,
|
||||
mce.getArgument(0), mce.getArgument(1), "MQ04", "MQ05");
|
||||
}
|
||||
|
||||
private WritePoint tryDetectKafka(MethodCallExpr mce, String filePath) {
|
||||
if (!"send".equals(mce.getNameAsString()) || !isKafkaScope(mce)) {
|
||||
return null;
|
||||
}
|
||||
int argc = mce.getArguments().size();
|
||||
if (argc == 1 && enabledPatterns.contains("MQ-K03")) {
|
||||
return buildFromProducerRecord(mce, filePath);
|
||||
}
|
||||
String basePattern;
|
||||
Expression topicArg;
|
||||
Expression payloadArg;
|
||||
if (argc == 2 && enabledPatterns.contains("MQ-K01")) {
|
||||
basePattern = "MQ-K01";
|
||||
topicArg = mce.getArgument(0);
|
||||
payloadArg = mce.getArgument(1);
|
||||
} else if (argc >= 3 && enabledPatterns.contains("MQ-K02")) {
|
||||
basePattern = "MQ-K02";
|
||||
topicArg = mce.getArgument(0);
|
||||
payloadArg = mce.getArgument(argc - 1);
|
||||
} else {
|
||||
return null;
|
||||
}
|
||||
return buildMqWritePoint(mce, filePath, basePattern, WritePoint.CHANNEL_KAFKA,
|
||||
topicArg, payloadArg, null, "MQ-K04");
|
||||
}
|
||||
|
||||
private WritePoint buildFromProducerRecord(MethodCallExpr mce, String filePath) {
|
||||
Expression recordArg = mce.getArgument(0);
|
||||
ObjectCreationExpr creation = findProducerRecordCreation(recordArg, mce);
|
||||
if (creation == null || creation.getArguments().size() < 2) {
|
||||
return null;
|
||||
}
|
||||
Expression topicArg = creation.getArgument(0);
|
||||
Expression payloadArg = creation.getArgument(creation.getArguments().size() - 1);
|
||||
WritePoint wp = buildMqWritePoint(mce, filePath, "MQ-K03", WritePoint.CHANNEL_KAFKA,
|
||||
topicArg, payloadArg, null, "MQ-K04");
|
||||
if (wp != null) {
|
||||
wp.setValueExpression(recordArg.toString());
|
||||
}
|
||||
return wp;
|
||||
}
|
||||
|
||||
private ObjectCreationExpr findProducerRecordCreation(Expression expr, MethodCallExpr contextCall) {
|
||||
if (expr instanceof ObjectCreationExpr) {
|
||||
ObjectCreationExpr oce = (ObjectCreationExpr) expr;
|
||||
if ("ProducerRecord".equals(oce.getType().getNameAsString())) {
|
||||
return oce;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
if (expr instanceof NameExpr) {
|
||||
String name = ((NameExpr) expr).getNameAsString();
|
||||
Optional<CallableDeclaration> callable = contextCall.findAncestor(CallableDeclaration.class);
|
||||
if (!callable.isPresent()) {
|
||||
return null;
|
||||
}
|
||||
for (VariableDeclarator var : callable.get().findAll(VariableDeclarator.class)) {
|
||||
if (!var.getNameAsString().equals(name) || !var.getInitializer().isPresent()) {
|
||||
continue;
|
||||
}
|
||||
Expression init = var.getInitializer().get();
|
||||
if (init instanceof ObjectCreationExpr
|
||||
&& "ProducerRecord".equals(((ObjectCreationExpr) init).getType().getNameAsString())) {
|
||||
return (ObjectCreationExpr) init;
|
||||
}
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private WritePoint buildMqWritePoint(MethodCallExpr mce, String filePath, String basePattern,
|
||||
String channel, Expression destArg, Expression payloadArg,
|
||||
String messagePattern, String jsonPattern) {
|
||||
PayloadResolution resolved = resolvePayload(payloadArg, mce, messagePattern, jsonPattern);
|
||||
if (resolved == null) {
|
||||
return null;
|
||||
}
|
||||
String pattern = resolved.overridePattern != null ? resolved.overridePattern : basePattern;
|
||||
if (!enabledPatterns.contains(pattern)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
WritePoint wp = new WritePoint();
|
||||
wp.setFilePath(filePath);
|
||||
wp.setLineNumber(mce.getBegin().map(p -> p.line).orElse(0));
|
||||
wp.setPattern(pattern);
|
||||
wp.setChannel(channel);
|
||||
wp.setKeyExpression(destArg.toString());
|
||||
wp.setValueExpression(payloadArg.toString());
|
||||
fillEnclosing(mce, wp);
|
||||
|
||||
SourceIndex.IndexedType context = index.get(wp.getEnclosingClass());
|
||||
ClassOrInterfaceDeclaration enclosingDecl = mce
|
||||
.findAncestor(ClassOrInterfaceDeclaration.class).orElse(null);
|
||||
wp.setResolvedKeyPattern(keyResolver.resolve(destArg, enclosingDecl, context));
|
||||
|
||||
InferredType inferred = inferType(resolved.typeExpr, mce, context);
|
||||
if (inferred != null) {
|
||||
wp.setResolvedValueType(inferred.fqn);
|
||||
wp.setRootArray(inferred.isArray);
|
||||
wp.setConfidence(inferred.fqn == null ? 0.4 : 1.0);
|
||||
} else {
|
||||
wp.setConfidence(0.4);
|
||||
}
|
||||
return wp;
|
||||
}
|
||||
|
||||
/**
|
||||
* 解析 payload:Message/MessageBuilder(MQ04)、JSON 字符串(MQ05/MQ-K04)、直传对象。
|
||||
* 无法解析的 Message/纯 String 返回 null(忽略)。
|
||||
*/
|
||||
private PayloadResolution resolvePayload(Expression payloadArg, MethodCallExpr mce,
|
||||
String messagePattern, String jsonPattern) {
|
||||
if (payloadArg == null) {
|
||||
return null;
|
||||
}
|
||||
// MQ04:Message<T> 泛型 / MessageBuilder.withPayload
|
||||
if (messagePattern != null && enabledPatterns.contains(messagePattern)) {
|
||||
Expression fromMessage = unwrapMessagePayload(payloadArg, mce);
|
||||
if (fromMessage != null) {
|
||||
return new PayloadResolution(fromMessage, messagePattern);
|
||||
}
|
||||
if (isEnvelopeBare(payloadArg, mce)) {
|
||||
return null;
|
||||
}
|
||||
} else if (isEnvelopeBare(payloadArg, mce)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// 内联 JSON 序列化
|
||||
Expression serialized = unwrapSerializer(payloadArg);
|
||||
if (serialized != null) {
|
||||
if (jsonPattern != null && enabledPatterns.contains(jsonPattern)) {
|
||||
return new PayloadResolution(serialized, jsonPattern);
|
||||
}
|
||||
return new PayloadResolution(serialized, null);
|
||||
}
|
||||
|
||||
// 局部 String = toJSONString(...)
|
||||
if (jsonPattern != null && enabledPatterns.contains(jsonPattern)) {
|
||||
Expression fromLocalJson = unwrapJsonStringVar(payloadArg, mce);
|
||||
if (fromLocalJson != null) {
|
||||
return new PayloadResolution(fromLocalJson, jsonPattern);
|
||||
}
|
||||
}
|
||||
|
||||
if (isTrivialValue(payloadArg) || isBareStringOrBytesPayload(payloadArg, mce)) {
|
||||
return null;
|
||||
}
|
||||
return new PayloadResolution(payloadArg, null);
|
||||
}
|
||||
|
||||
private Expression unwrapMessagePayload(Expression payloadArg, MethodCallExpr mce) {
|
||||
Expression fromBuilder = unwrapMessageBuilderPayload(payloadArg, mce);
|
||||
if (fromBuilder != null) {
|
||||
return fromBuilder;
|
||||
}
|
||||
// Message<DutyImNotice> → DutyImNotice(无 withPayload 也可)
|
||||
if (payloadArg instanceof NameExpr) {
|
||||
Type declared = findVariableType(((NameExpr) payloadArg).getNameAsString(), mce);
|
||||
if (declared instanceof ClassOrInterfaceType) {
|
||||
ClassOrInterfaceType cit = (ClassOrInterfaceType) declared;
|
||||
if (ENVELOPE_TYPES.contains(cit.getNameAsString())) {
|
||||
Optional<Type> payloadType = cit.getTypeArguments()
|
||||
.filter(a -> !a.isEmpty())
|
||||
.map(a -> a.get(a.size() - 1));
|
||||
if (payloadType.isPresent() && payloadType.get() instanceof ClassOrInterfaceType) {
|
||||
// 用伪 ObjectCreation 表达类型不便;改为在 infer 前用 NameExpr 不够
|
||||
// 返回一个标记:借助 CastExpr 包装类型信息
|
||||
return new CastExpr(payloadType.get(), payloadArg);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private Expression unwrapMessageBuilderPayload(Expression payloadArg, MethodCallExpr mce) {
|
||||
Expression chain = payloadArg;
|
||||
if (payloadArg instanceof NameExpr) {
|
||||
String name = ((NameExpr) payloadArg).getNameAsString();
|
||||
Optional<CallableDeclaration> callable = mce.findAncestor(CallableDeclaration.class);
|
||||
if (callable.isPresent()) {
|
||||
for (VariableDeclarator var : callable.get().findAll(VariableDeclarator.class)) {
|
||||
if (var.getNameAsString().equals(name) && var.getInitializer().isPresent()) {
|
||||
chain = var.getInitializer().get();
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return findWithPayloadArg(chain);
|
||||
}
|
||||
|
||||
private Expression findWithPayloadArg(Expression expr) {
|
||||
Expression current = expr;
|
||||
int guard = 0;
|
||||
while (current instanceof MethodCallExpr && guard++ < 16) {
|
||||
MethodCallExpr call = (MethodCallExpr) current;
|
||||
if ("withPayload".equals(call.getNameAsString()) && !call.getArguments().isEmpty()) {
|
||||
return call.getArgument(0);
|
||||
}
|
||||
if (!call.getScope().isPresent()) {
|
||||
break;
|
||||
}
|
||||
current = call.getScope().get();
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private Expression unwrapJsonStringVar(Expression payloadArg, MethodCallExpr mce) {
|
||||
if (!(payloadArg instanceof NameExpr)) {
|
||||
return null;
|
||||
}
|
||||
String name = ((NameExpr) payloadArg).getNameAsString();
|
||||
Type declared = findVariableType(name, mce);
|
||||
if (declared != null) {
|
||||
String simple = declared.isClassOrInterfaceType()
|
||||
? declared.asClassOrInterfaceType().getNameAsString()
|
||||
: declared.asString();
|
||||
if (!"String".equals(simple)) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
Optional<CallableDeclaration> callable = mce.findAncestor(CallableDeclaration.class);
|
||||
if (!callable.isPresent()) {
|
||||
return null;
|
||||
}
|
||||
for (VariableDeclarator var : callable.get().findAll(VariableDeclarator.class)) {
|
||||
if (!var.getNameAsString().equals(name) || !var.getInitializer().isPresent()) {
|
||||
continue;
|
||||
}
|
||||
return unwrapSerializer(var.getInitializer().get());
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private boolean isEnvelopeBare(Expression typeExpr, MethodCallExpr contextCall) {
|
||||
if (!(typeExpr instanceof NameExpr)) {
|
||||
return false;
|
||||
}
|
||||
Type declared = findVariableType(((NameExpr) typeExpr).getNameAsString(), contextCall);
|
||||
if (declared == null) {
|
||||
return false;
|
||||
}
|
||||
String simple = declared.isClassOrInterfaceType()
|
||||
? declared.asClassOrInterfaceType().getNameAsString()
|
||||
: declared.asString();
|
||||
return ENVELOPE_TYPES.contains(simple);
|
||||
}
|
||||
|
||||
private boolean isRocketMqScope(MethodCallExpr mce) {
|
||||
String scope = mce.getScope().map(Expression::toString).orElse("").toLowerCase();
|
||||
return scope.contains("rocketmqtemplate") || scope.contains("rocketmq");
|
||||
}
|
||||
|
||||
private boolean isKafkaScope(MethodCallExpr mce) {
|
||||
String scope = mce.getScope().map(Expression::toString).orElse("").toLowerCase();
|
||||
return scope.contains("kafkatemplate") || "kafkatemplate".equals(scope);
|
||||
}
|
||||
|
||||
private boolean isBareStringOrBytesPayload(Expression typeExpr, MethodCallExpr contextCall) {
|
||||
if (typeExpr instanceof StringLiteralExpr) {
|
||||
return true;
|
||||
}
|
||||
if (typeExpr instanceof NameExpr) {
|
||||
Type declared = findVariableType(((NameExpr) typeExpr).getNameAsString(), contextCall);
|
||||
if (declared != null) {
|
||||
String simple = declared.isClassOrInterfaceType()
|
||||
? declared.asClassOrInterfaceType().getNameAsString()
|
||||
: declared.asString();
|
||||
if (IGNORE_BARE_TYPES.contains(simple) || "byte[]".equals(declared.asString())) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private boolean isTrivialValue(Expression expr) {
|
||||
if (expr instanceof StringLiteralExpr
|
||||
|| expr instanceof IntegerLiteralExpr
|
||||
|| expr instanceof LongLiteralExpr
|
||||
|| expr instanceof BooleanLiteralExpr
|
||||
|| expr instanceof NullLiteralExpr) {
|
||||
return true;
|
||||
}
|
||||
if (expr instanceof BinaryExpr) {
|
||||
BinaryExpr be = (BinaryExpr) expr;
|
||||
if (be.getOperator() == BinaryExpr.Operator.PLUS) {
|
||||
return isTrivialValue(be.getLeft()) && isTrivialValue(be.getRight());
|
||||
}
|
||||
}
|
||||
if (expr instanceof MethodCallExpr) {
|
||||
MethodCallExpr call = (MethodCallExpr) expr;
|
||||
String name = call.getNameAsString();
|
||||
if (TRIVIAL_VALUE_CALLS.contains(name)) {
|
||||
return true;
|
||||
}
|
||||
if ("valueOf".equals(name) && !call.getArguments().isEmpty()) {
|
||||
return isTrivialValue(call.getArgument(0));
|
||||
}
|
||||
}
|
||||
if (expr instanceof ObjectCreationExpr) {
|
||||
return "UUID".equals(((ObjectCreationExpr) expr).getType().getNameAsString());
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private Expression unwrapSerializer(Expression valueArg) {
|
||||
if (valueArg instanceof MethodCallExpr) {
|
||||
MethodCallExpr call = (MethodCallExpr) valueArg;
|
||||
if (SERIALIZE_METHODS.contains(call.getNameAsString()) && !call.getArguments().isEmpty()) {
|
||||
return call.getArgument(0);
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private void fillEnclosing(MethodCallExpr mce, WritePoint wp) {
|
||||
Optional<ClassOrInterfaceDeclaration> clazz = mce.findAncestor(ClassOrInterfaceDeclaration.class);
|
||||
wp.setEnclosingClass(clazz.map(d -> d.getFullyQualifiedName().orElse(d.getNameAsString()))
|
||||
.orElse("<unknown>"));
|
||||
Optional<CallableDeclaration> method = mce.findAncestor(CallableDeclaration.class);
|
||||
wp.setEnclosingMethod(method.map(CallableDeclaration::getNameAsString).orElse("<unknown>"));
|
||||
}
|
||||
|
||||
private InferredType inferType(Expression expr, MethodCallExpr contextCall,
|
||||
SourceIndex.IndexedType context) {
|
||||
if (expr instanceof CastExpr) {
|
||||
return resolveTypeNode(((CastExpr) expr).getType(), context);
|
||||
}
|
||||
if (expr instanceof ObjectCreationExpr) {
|
||||
return resolveTypeNode(((ObjectCreationExpr) expr).getType(), context);
|
||||
}
|
||||
if (expr instanceof NameExpr) {
|
||||
String name = ((NameExpr) expr).getNameAsString();
|
||||
Type declared = findVariableType(name, contextCall);
|
||||
if (declared != null) {
|
||||
return resolveTypeNode(declared, context);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
if (expr instanceof MethodCallExpr) {
|
||||
MethodCallExpr call = (MethodCallExpr) expr;
|
||||
Optional<ClassOrInterfaceDeclaration> clazz = contextCall
|
||||
.findAncestor(ClassOrInterfaceDeclaration.class);
|
||||
if (clazz.isPresent()) {
|
||||
for (MethodDeclaration md : clazz.get().getMethods()) {
|
||||
if (md.getNameAsString().equals(call.getNameAsString())) {
|
||||
return resolveTypeNode(md.getType(), context);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private Type findVariableType(String name, MethodCallExpr contextCall) {
|
||||
Optional<CallableDeclaration> callable = contextCall.findAncestor(CallableDeclaration.class);
|
||||
if (callable.isPresent()) {
|
||||
CallableDeclaration<?> decl = callable.get();
|
||||
for (VariableDeclarator var : decl.findAll(VariableDeclarator.class)) {
|
||||
if (var.getNameAsString().equals(name)) {
|
||||
return var.getType();
|
||||
}
|
||||
}
|
||||
for (Parameter p : decl.getParameters()) {
|
||||
if (p.getNameAsString().equals(name)) {
|
||||
return p.getType();
|
||||
}
|
||||
}
|
||||
}
|
||||
Optional<ClassOrInterfaceDeclaration> clazz = contextCall.findAncestor(ClassOrInterfaceDeclaration.class);
|
||||
if (clazz.isPresent()) {
|
||||
for (FieldDeclaration field : clazz.get().getFields()) {
|
||||
for (VariableDeclarator var : field.getVariables()) {
|
||||
if (var.getNameAsString().equals(name)) {
|
||||
return var.getType();
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private InferredType resolveTypeNode(Type type, SourceIndex.IndexedType context) {
|
||||
if (!(type instanceof ClassOrInterfaceType)) {
|
||||
return new InferredType(null, false);
|
||||
}
|
||||
ClassOrInterfaceType cit = (ClassOrInterfaceType) type;
|
||||
String simple = cit.getNameAsString();
|
||||
if (COLLECTION_SIMPLE.contains(simple)) {
|
||||
Optional<Type> arg = cit.getTypeArguments().filter(a -> !a.isEmpty()).map(a -> a.get(0));
|
||||
if (arg.isPresent() && arg.get() instanceof ClassOrInterfaceType) {
|
||||
String elementFqn = resolveFqn((ClassOrInterfaceType) arg.get(), context);
|
||||
return new InferredType(elementFqn, true);
|
||||
}
|
||||
return new InferredType(null, true);
|
||||
}
|
||||
if (ENVELOPE_TYPES.contains(simple)) {
|
||||
Optional<Type> payload = cit.getTypeArguments()
|
||||
.filter(a -> !a.isEmpty())
|
||||
.map(a -> a.get(a.size() - 1));
|
||||
if (payload.isPresent()) {
|
||||
return resolveTypeNode(payload.get(), context);
|
||||
}
|
||||
return new InferredType(null, false);
|
||||
}
|
||||
return new InferredType(resolveFqn(cit, context), false);
|
||||
}
|
||||
|
||||
private String resolveFqn(ClassOrInterfaceType cit, SourceIndex.IndexedType context) {
|
||||
String fqn = index.resolveFqn(cit.getNameWithScope(), context);
|
||||
if (fqn == null) {
|
||||
fqn = index.resolveFqn(cit.getNameAsString(), context);
|
||||
}
|
||||
return fqn;
|
||||
}
|
||||
|
||||
private static final class PayloadResolution {
|
||||
final Expression typeExpr;
|
||||
final String overridePattern;
|
||||
|
||||
PayloadResolution(Expression typeExpr, String overridePattern) {
|
||||
this.typeExpr = typeExpr;
|
||||
this.overridePattern = overridePattern;
|
||||
}
|
||||
}
|
||||
|
||||
private static final class InferredType {
|
||||
final String fqn;
|
||||
final boolean isArray;
|
||||
|
||||
InferredType(String fqn, boolean isArray) {
|
||||
this.fqn = fqn;
|
||||
this.isArray = isArray;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,10 +1,14 @@
|
||||
package com.codechecker.cache.detector;
|
||||
|
||||
/**
|
||||
* 一个 Redis value 写入点的静态描述。
|
||||
* 一个 Redis / MQ value 写入(投递)点的静态描述。
|
||||
*/
|
||||
public class WritePoint {
|
||||
|
||||
public static final String CHANNEL_REDIS = "REDIS";
|
||||
public static final String CHANNEL_ROCKETMQ = "ROCKETMQ";
|
||||
public static final String CHANNEL_KAFKA = "KAFKA";
|
||||
|
||||
private String filePath;
|
||||
private int lineNumber;
|
||||
private String enclosingClass;
|
||||
@@ -20,6 +24,9 @@ public class WritePoint {
|
||||
|
||||
private double confidence = 1.0;
|
||||
|
||||
/** REDIS / ROCKETMQ / KAFKA;默认 REDIS 兼容现网 */
|
||||
private String channel = CHANNEL_REDIS;
|
||||
|
||||
/** 稳定标识:用于在 old/new 两个版本间配对同一写入点 */
|
||||
public String signature() {
|
||||
return enclosingClass + "#" + enclosingMethod + "|" + normalizeKey();
|
||||
@@ -29,6 +36,10 @@ public class WritePoint {
|
||||
return keyExpression == null ? "" : keyExpression.replaceAll("\\s+", "");
|
||||
}
|
||||
|
||||
public boolean isMq() {
|
||||
return CHANNEL_ROCKETMQ.equals(channel) || CHANNEL_KAFKA.equals(channel);
|
||||
}
|
||||
|
||||
public String getFilePath() {
|
||||
return filePath;
|
||||
}
|
||||
@@ -117,6 +128,14 @@ public class WritePoint {
|
||||
this.confidence = confidence;
|
||||
}
|
||||
|
||||
public String getChannel() {
|
||||
return channel;
|
||||
}
|
||||
|
||||
public void setChannel(String channel) {
|
||||
this.channel = channel == null || channel.isEmpty() ? CHANNEL_REDIS : channel;
|
||||
}
|
||||
|
||||
public String location() {
|
||||
String simpleClass = enclosingClass;
|
||||
if (simpleClass != null && simpleClass.contains(".")) {
|
||||
|
||||
@@ -7,6 +7,7 @@ public enum ChangeType {
|
||||
FIELD_REMOVED(Severity.P0, "字段删除"),
|
||||
TYPE_CHANGED(Severity.P0, "字段类型变更"),
|
||||
WRAPPER_ADDED(Severity.P0, "新增包装层"),
|
||||
WRAPPER_REMOVED(Severity.P0, "删除包装层"),
|
||||
FIELD_PATH_MOVED(Severity.P0, "字段路径迁移"),
|
||||
FIELD_ADDED(Severity.P1, "新增字段"),
|
||||
KEY_PATTERN_CHANGED(Severity.P1, "Key 模式变更"),
|
||||
|
||||
@@ -22,40 +22,57 @@ public class SchemaDiffer {
|
||||
public List<SchemaChange> diff(TypeSchema oldSchema, TypeSchema newSchema) {
|
||||
List<SchemaChange> changes = new ArrayList<>();
|
||||
|
||||
Map<String, JsonType> oldLeaves = leaves(oldSchema);
|
||||
Map<String, JsonType> newLeaves = leaves(newSchema);
|
||||
Map<String, FieldSchema> oldAll = fieldsByPath(oldSchema);
|
||||
Map<String, FieldSchema> newAll = fieldsByPath(newSchema);
|
||||
|
||||
Map<String, FieldSchema> oldLeaves = leaves(oldSchema);
|
||||
Map<String, FieldSchema> newLeaves = leaves(newSchema);
|
||||
|
||||
Set<String> typeChangedPaths = new LinkedHashSet<>();
|
||||
|
||||
// 类型变更:同路径下 JsonType 或 Java 类型任一变化(覆盖 BigDecimal→String 等同桶变更)
|
||||
for (String path : oldAll.keySet()) {
|
||||
FieldSchema oldField = oldAll.get(path);
|
||||
FieldSchema newField = newAll.get(path);
|
||||
if (newField == null) {
|
||||
continue;
|
||||
}
|
||||
if (!isTypeChange(oldField, newField)) {
|
||||
continue;
|
||||
}
|
||||
typeChangedPaths.add(path);
|
||||
String oldLabel = typeLabel(oldField);
|
||||
String newLabel = typeLabel(newField);
|
||||
SchemaChange c = new SchemaChange(ChangeType.TYPE_CHANGED);
|
||||
c.setFieldPath(path);
|
||||
c.setOldValue(oldLabel);
|
||||
c.setNewValue(newLabel);
|
||||
c.setMessage("字段 " + path + " 类型由 " + oldLabel + " 变为 " + newLabel);
|
||||
changes.add(c);
|
||||
}
|
||||
|
||||
Set<String> removed = new LinkedHashSet<>(oldLeaves.keySet());
|
||||
removed.removeAll(newLeaves.keySet());
|
||||
removed.removeAll(typeChangedPaths);
|
||||
Set<String> added = new LinkedHashSet<>(newLeaves.keySet());
|
||||
added.removeAll(oldLeaves.keySet());
|
||||
added.removeAll(typeChangedPaths);
|
||||
|
||||
// 类型变更(同路径)
|
||||
for (String path : oldLeaves.keySet()) {
|
||||
if (newLeaves.containsKey(path)) {
|
||||
JsonType oldType = oldLeaves.get(path);
|
||||
JsonType newType = newLeaves.get(path);
|
||||
if (oldType != newType) {
|
||||
SchemaChange c = new SchemaChange(ChangeType.TYPE_CHANGED);
|
||||
c.setFieldPath(path);
|
||||
c.setOldValue(oldType.name());
|
||||
c.setNewValue(newType.name());
|
||||
c.setMessage("字段 " + path + " 类型由 " + oldType + " 变为 " + newType);
|
||||
changes.add(c);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 路径迁移检测(removed 的路径是某 added 路径的后缀)
|
||||
// 路径迁移:
|
||||
// - 下沉(加包装):旧路径是新路径后缀,如 dbName → vo.dbName
|
||||
// - 上提(拆包装):新路径是旧路径后缀,如 vo.dbName → dbName
|
||||
List<String[]> moves = new ArrayList<>();
|
||||
Set<String> matchedRemoved = new LinkedHashSet<>();
|
||||
Set<String> matchedAdded = new LinkedHashSet<>();
|
||||
for (String r : removed) {
|
||||
if (matchedRemoved.contains(r)) {
|
||||
continue;
|
||||
}
|
||||
for (String a : added) {
|
||||
if (matchedAdded.contains(a)) {
|
||||
continue;
|
||||
}
|
||||
if (isSuffix(a, r)) {
|
||||
if (isSuffix(a, r) || isSuffix(r, a)) {
|
||||
moves.add(new String[]{r, a});
|
||||
matchedRemoved.add(r);
|
||||
matchedAdded.add(a);
|
||||
@@ -64,16 +81,16 @@ public class SchemaDiffer {
|
||||
}
|
||||
}
|
||||
|
||||
// 包装层检测:多个迁移共享同一新前缀
|
||||
Map<String, Integer> prefixCount = new LinkedHashMap<>();
|
||||
// 新增包装层:多个迁移共享同一新前缀
|
||||
Map<String, Integer> newPrefixCount = new LinkedHashMap<>();
|
||||
for (String[] move : moves) {
|
||||
String a = move[1];
|
||||
if (a.contains(".")) {
|
||||
String prefix = a.substring(0, a.indexOf('.'));
|
||||
prefixCount.merge(prefix, 1, Integer::sum);
|
||||
newPrefixCount.merge(prefix, 1, Integer::sum);
|
||||
}
|
||||
}
|
||||
for (Map.Entry<String, Integer> e : prefixCount.entrySet()) {
|
||||
for (Map.Entry<String, Integer> e : newPrefixCount.entrySet()) {
|
||||
if (e.getValue() >= 2) {
|
||||
SchemaChange c = new SchemaChange(ChangeType.WRAPPER_ADDED);
|
||||
c.setFieldPath(e.getKey());
|
||||
@@ -83,6 +100,25 @@ public class SchemaDiffer {
|
||||
}
|
||||
}
|
||||
|
||||
// 删除包装层:多个迁移共享同一旧前缀
|
||||
Map<String, Integer> oldPrefixCount = new LinkedHashMap<>();
|
||||
for (String[] move : moves) {
|
||||
String r = move[0];
|
||||
if (r.contains(".")) {
|
||||
String prefix = r.substring(0, r.indexOf('.'));
|
||||
oldPrefixCount.merge(prefix, 1, Integer::sum);
|
||||
}
|
||||
}
|
||||
for (Map.Entry<String, Integer> e : oldPrefixCount.entrySet()) {
|
||||
if (e.getValue() >= 2) {
|
||||
SchemaChange c = new SchemaChange(ChangeType.WRAPPER_REMOVED);
|
||||
c.setFieldPath(e.getKey());
|
||||
c.setOldValue(e.getKey());
|
||||
c.setMessage("删除包装层 " + e.getKey() + ",其下字段被提升为外层(影响 " + e.getValue() + " 个字段)");
|
||||
changes.add(c);
|
||||
}
|
||||
}
|
||||
|
||||
for (String[] move : moves) {
|
||||
SchemaChange c = new SchemaChange(ChangeType.FIELD_PATH_MOVED);
|
||||
c.setFieldPath(move[1]);
|
||||
@@ -99,7 +135,7 @@ public class SchemaDiffer {
|
||||
}
|
||||
SchemaChange c = new SchemaChange(ChangeType.FIELD_REMOVED);
|
||||
c.setFieldPath(r);
|
||||
c.setOldValue(oldLeaves.get(r).name());
|
||||
c.setOldValue(typeLabel(oldLeaves.get(r)));
|
||||
c.setMessage("删除字段 " + r);
|
||||
changes.add(c);
|
||||
}
|
||||
@@ -111,7 +147,7 @@ public class SchemaDiffer {
|
||||
}
|
||||
SchemaChange c = new SchemaChange(ChangeType.FIELD_ADDED);
|
||||
c.setFieldPath(a);
|
||||
c.setNewValue(newLeaves.get(a).name());
|
||||
c.setNewValue(typeLabel(newLeaves.get(a)));
|
||||
c.setMessage("新增字段 " + a);
|
||||
changes.add(c);
|
||||
}
|
||||
@@ -119,11 +155,104 @@ public class SchemaDiffer {
|
||||
return changes;
|
||||
}
|
||||
|
||||
private Map<String, JsonType> leaves(TypeSchema schema) {
|
||||
Map<String, JsonType> result = new LinkedHashMap<>();
|
||||
private boolean isTypeChange(FieldSchema oldField, FieldSchema newField) {
|
||||
if (oldField.getJsonType() != newField.getJsonType()) {
|
||||
return true;
|
||||
}
|
||||
String oldJava = normalizeJavaType(oldField.getJavaType());
|
||||
String newJava = normalizeJavaType(newField.getJavaType());
|
||||
if (oldJava.isEmpty() || newJava.isEmpty()) {
|
||||
return false;
|
||||
}
|
||||
return !oldJava.equals(newJava);
|
||||
}
|
||||
|
||||
/**
|
||||
* 展示用类型标签:优先 Java 简单名,并附带 JsonType(便于识别同桶变更)。
|
||||
*/
|
||||
private String typeLabel(FieldSchema field) {
|
||||
if (field == null) {
|
||||
return "?";
|
||||
}
|
||||
String java = simpleJavaType(field.getJavaType());
|
||||
JsonType json = field.getJsonType();
|
||||
if (java.isEmpty() || "?".equals(java)) {
|
||||
return json == null ? "?" : json.name();
|
||||
}
|
||||
if (json == null) {
|
||||
return java;
|
||||
}
|
||||
return java + "/" + json.name();
|
||||
}
|
||||
|
||||
/**
|
||||
* 归一化后用于相等比较:去掉泛型与包名,基本类型与包装类型视为同一序列化类型。
|
||||
*/
|
||||
static String normalizeJavaType(String javaType) {
|
||||
String simple = simpleJavaType(javaType);
|
||||
if (simple.isEmpty() || "?".equals(simple)) {
|
||||
return "";
|
||||
}
|
||||
switch (simple) {
|
||||
case "int":
|
||||
return "Integer";
|
||||
case "long":
|
||||
return "Long";
|
||||
case "short":
|
||||
return "Short";
|
||||
case "byte":
|
||||
return "Byte";
|
||||
case "double":
|
||||
return "Double";
|
||||
case "float":
|
||||
return "Float";
|
||||
case "boolean":
|
||||
return "Boolean";
|
||||
case "char":
|
||||
return "Character";
|
||||
default:
|
||||
return simple;
|
||||
}
|
||||
}
|
||||
|
||||
private static String simpleJavaType(String javaType) {
|
||||
if (javaType == null) {
|
||||
return "";
|
||||
}
|
||||
String s = javaType.trim();
|
||||
if (s.isEmpty()) {
|
||||
return "";
|
||||
}
|
||||
int lt = s.indexOf('<');
|
||||
if (lt > 0) {
|
||||
s = s.substring(0, lt).trim();
|
||||
}
|
||||
int dot = s.lastIndexOf('.');
|
||||
if (dot >= 0 && dot < s.length() - 1) {
|
||||
s = s.substring(dot + 1);
|
||||
}
|
||||
return s;
|
||||
}
|
||||
|
||||
private Map<String, FieldSchema> fieldsByPath(TypeSchema schema) {
|
||||
Map<String, FieldSchema> result = new LinkedHashMap<>();
|
||||
if (schema == null) {
|
||||
return result;
|
||||
}
|
||||
for (FieldSchema f : schema.getFields().values()) {
|
||||
result.put(f.getPath(), f);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private Map<String, FieldSchema> leaves(TypeSchema schema) {
|
||||
Map<String, FieldSchema> result = new LinkedHashMap<>();
|
||||
if (schema == null) {
|
||||
return result;
|
||||
}
|
||||
for (FieldSchema f : schema.getFields().values()) {
|
||||
if (f.getJsonType() != JsonType.OBJECT && f.getJsonType() != JsonType.ARRAY) {
|
||||
result.put(f.getPath(), f.getJsonType());
|
||||
result.put(f.getPath(), f);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
package com.codechecker.cache.key;
|
||||
|
||||
import com.codechecker.cache.schema.SourceIndex;
|
||||
import com.github.javaparser.Position;
|
||||
import com.github.javaparser.ast.body.ClassOrInterfaceDeclaration;
|
||||
import com.github.javaparser.ast.body.FieldDeclaration;
|
||||
import com.github.javaparser.ast.body.MethodDeclaration;
|
||||
import com.github.javaparser.ast.body.VariableDeclarator;
|
||||
import com.github.javaparser.ast.expr.AssignExpr;
|
||||
import com.github.javaparser.ast.expr.BinaryExpr;
|
||||
import com.github.javaparser.ast.expr.Expression;
|
||||
import com.github.javaparser.ast.expr.FieldAccessExpr;
|
||||
@@ -17,6 +19,8 @@ import java.util.Optional;
|
||||
|
||||
/**
|
||||
* 尽力将 Redis key 表达式静态推断为一个「key 模式」,动态部分以 {@code *} 表示。
|
||||
* <p>
|
||||
* 对局部变量名:先查同名静态常量 / 同名方法 return,再回溯方法内使用点之前最近一次声明或赋值。
|
||||
*/
|
||||
public class RedisKeyResolver {
|
||||
|
||||
@@ -52,12 +56,17 @@ public class RedisKeyResolver {
|
||||
}
|
||||
if (expr instanceof NameExpr) {
|
||||
String name = ((NameExpr) expr).getNameAsString();
|
||||
// 1) 静态常量 2) 同名方法 return 3) 局部变量定值(不改变既有优先级)
|
||||
String constVal = lookupConstant(enclosingClass, name, context, depth);
|
||||
if (constVal != null) {
|
||||
return constVal;
|
||||
}
|
||||
String methodVal = lookupMethodReturn(enclosingClass, name, context, depth);
|
||||
return methodVal != null ? methodVal : "*";
|
||||
if (methodVal != null) {
|
||||
return methodVal;
|
||||
}
|
||||
String localVal = lookupLocalBinding(expr, name, enclosingClass, context, depth);
|
||||
return localVal != null ? localVal : "*";
|
||||
}
|
||||
if (expr instanceof FieldAccessExpr) {
|
||||
FieldAccessExpr fae = (FieldAccessExpr) expr;
|
||||
@@ -94,6 +103,76 @@ public class RedisKeyResolver {
|
||||
return "*";
|
||||
}
|
||||
|
||||
/**
|
||||
* 在包围方法内,取使用点之前对 {@code name} 最近一次声明初始化或赋值的右侧表达式。
|
||||
*/
|
||||
private String lookupLocalBinding(Expression useSite, String name,
|
||||
ClassOrInterfaceDeclaration enclosingClass,
|
||||
SourceIndex.IndexedType context, int depth) {
|
||||
if (useSite == null || name == null || name.isEmpty()) {
|
||||
return null;
|
||||
}
|
||||
MethodDeclaration method = useSite.findAncestor(MethodDeclaration.class).orElse(null);
|
||||
if (method == null || !method.getBody().isPresent()) {
|
||||
return null;
|
||||
}
|
||||
|
||||
Position usePos = useSite.getBegin().orElse(null);
|
||||
Expression bestRhs = null;
|
||||
Position bestPos = null;
|
||||
|
||||
for (VariableDeclarator var : method.findAll(VariableDeclarator.class)) {
|
||||
if (!name.equals(var.getNameAsString()) || !var.getInitializer().isPresent()) {
|
||||
continue;
|
||||
}
|
||||
Position pos = var.getBegin().orElse(null);
|
||||
if (!isUsableBinding(pos, usePos, bestPos)) {
|
||||
continue;
|
||||
}
|
||||
bestPos = pos;
|
||||
bestRhs = var.getInitializer().get();
|
||||
}
|
||||
|
||||
for (AssignExpr assign : method.findAll(AssignExpr.class)) {
|
||||
if (!(assign.getTarget() instanceof NameExpr)) {
|
||||
continue;
|
||||
}
|
||||
if (!name.equals(((NameExpr) assign.getTarget()).getNameAsString())) {
|
||||
continue;
|
||||
}
|
||||
Position pos = assign.getBegin().orElse(null);
|
||||
if (!isUsableBinding(pos, usePos, bestPos)) {
|
||||
continue;
|
||||
}
|
||||
bestPos = pos;
|
||||
bestRhs = assign.getValue();
|
||||
}
|
||||
|
||||
if (bestRhs == null) {
|
||||
return null;
|
||||
}
|
||||
// 避免自引用(String key = key)陷入无意义递归
|
||||
if (bestRhs instanceof NameExpr
|
||||
&& name.equals(((NameExpr) bestRhs).getNameAsString())) {
|
||||
return null;
|
||||
}
|
||||
return resolveExpr(bestRhs, enclosingClass, context, depth + 1);
|
||||
}
|
||||
|
||||
/** 候选须严格在使用点之前,并在多个候选中取最近一次。 */
|
||||
private static boolean isUsableBinding(Position candidate, Position usePos, Position bestPos) {
|
||||
if (candidate == null) {
|
||||
return bestPos == null;
|
||||
}
|
||||
if (usePos != null && !candidate.isBefore(usePos)) {
|
||||
return false;
|
||||
}
|
||||
if (bestPos == null) {
|
||||
return true;
|
||||
}
|
||||
return candidate.isAfter(bestPos);
|
||||
}
|
||||
|
||||
private String lookupConstant(ClassOrInterfaceDeclaration clazz, String name,
|
||||
SourceIndex.IndexedType context, int depth) {
|
||||
if (clazz == null) {
|
||||
|
||||
@@ -7,19 +7,21 @@ import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* 按 Redis key(或未知 key 时的写入点)聚合后的结构变更摘要。
|
||||
* 按 Redis key / MQ Topic(或未知时的写入点)聚合后的结构变更摘要。
|
||||
*/
|
||||
public class KeyStructureChange {
|
||||
|
||||
/** 解析到的 key 模式;未知时为 unknown-key */
|
||||
/** 解析到的 key / destination 模式;未知时为 unknown-key */
|
||||
private String keyPattern;
|
||||
/** 源码中的 key 表达式,如 req.getKey() */
|
||||
/** 源码中的 key / destination 表达式 */
|
||||
private String keyExpression;
|
||||
/** 写入位置 Class#method:line */
|
||||
private String writeLocation;
|
||||
/** 展示用 value 类型,如 List<ClockInExportVo> */
|
||||
private String valueType;
|
||||
private boolean keyUnresolved;
|
||||
/** REDIS / ROCKETMQ / KAFKA */
|
||||
private String channel = "REDIS";
|
||||
private String oldSkeletonJson;
|
||||
private String newSkeletonJson;
|
||||
private Severity severity = Severity.P2;
|
||||
@@ -65,6 +67,28 @@ public class KeyStructureChange {
|
||||
this.keyUnresolved = keyUnresolved;
|
||||
}
|
||||
|
||||
public String getChannel() {
|
||||
return channel;
|
||||
}
|
||||
|
||||
public void setChannel(String channel) {
|
||||
this.channel = channel == null || channel.isEmpty() ? "REDIS" : channel;
|
||||
}
|
||||
|
||||
public boolean isMq() {
|
||||
return "ROCKETMQ".equals(channel) || "KAFKA".equals(channel);
|
||||
}
|
||||
|
||||
public String channelDisplay() {
|
||||
if ("ROCKETMQ".equals(channel)) {
|
||||
return "RocketMQ";
|
||||
}
|
||||
if ("KAFKA".equals(channel)) {
|
||||
return "Kafka";
|
||||
}
|
||||
return "Redis";
|
||||
}
|
||||
|
||||
public String getOldSkeletonJson() {
|
||||
return oldSkeletonJson;
|
||||
}
|
||||
@@ -102,7 +126,7 @@ public class KeyStructureChange {
|
||||
}
|
||||
}
|
||||
|
||||
/** 通知里 Key 行展示文本:未解析优先用表达式。 */
|
||||
/** 通知里 Key/Topic 行展示文本:未解析优先用表达式。 */
|
||||
public String displayKey() {
|
||||
if (keyUnresolved && keyExpression != null && !keyExpression.trim().isEmpty()) {
|
||||
return keyExpression.trim();
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
package com.codechecker.cache.report;
|
||||
|
||||
import com.codechecker.cache.diff.ChangeType;
|
||||
import com.codechecker.cache.diff.SchemaChange;
|
||||
import com.codechecker.cache.diff.Severity;
|
||||
|
||||
@@ -7,6 +8,7 @@ import java.nio.charset.StandardCharsets;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Collections;
|
||||
import java.util.EnumMap;
|
||||
import java.util.LinkedHashSet;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
@@ -14,7 +16,7 @@ import java.util.Set;
|
||||
/**
|
||||
* 将 {@link CheckReport} 渲染为企微 Markdown / 控制台文本。
|
||||
* <ul>
|
||||
* <li>企微:按 key 展示位置/类型/序列化骨架变更,不含字段明细,不分 P0/P1/P2</li>
|
||||
* <li>企微:按 key 展示位置/类型/序列化骨架变更;类型变更另附摘要行,不分 P0/P1/P2</li>
|
||||
* <li>未解析 key 展示源码表达式 + 灰色「key 无法解析」提示</li>
|
||||
* <li>多 key 优先拼成一条;超过企微上限则按 key 拆成多条</li>
|
||||
* <li>CI:先打字段明细,再完整输出企微 Markdown(拆分后的每条)</li>
|
||||
@@ -123,30 +125,127 @@ public class ReportBuilder {
|
||||
|
||||
private String renderKeyBlock(KeyStructureChange kc) {
|
||||
StringBuilder sb = new StringBuilder();
|
||||
appendKeyLine(sb, kc.displayKey(), kc.getKeyExpression(), kc.isKeyUnresolved());
|
||||
appendDestinationLine(sb, kc);
|
||||
if (kc.isMq()) {
|
||||
sb.append(" > **通道**: ").append(kc.channelDisplay()).append('\n');
|
||||
}
|
||||
appendMetaLines(sb, kc.getWriteLocation(), kc.getValueType());
|
||||
String oldJson = nvl(kc.getOldSkeletonJson());
|
||||
String newJson = nvl(kc.getNewSkeletonJson());
|
||||
Set<String> oldHighlight = SkeletonAnnotator.pathsForOldSkeleton(kc.getFieldDetails());
|
||||
Set<String> newHighlight = SkeletonAnnotator.pathsForNewSkeleton(kc.getFieldDetails());
|
||||
Set<String> wrappersRemoved = SkeletonAnnotator.pathsForWrapperRemoved(kc.getFieldDetails());
|
||||
Set<String> newGreen = SkeletonAnnotator.pathsForNewSkeleton(kc.getFieldDetails());
|
||||
Set<String> wrappersAdded = SkeletonAnnotator.pathsForWrapperAdded(kc.getFieldDetails());
|
||||
Set<String> typeGray = SkeletonAnnotator.pathsForTypeChanged(kc.getFieldDetails());
|
||||
String oldRendered = oldJson.isEmpty()
|
||||
? "" : "“" + SkeletonAnnotator.annotateOldForWecom(oldJson, oldHighlight) + "”";
|
||||
? "" : "“" + SkeletonAnnotator.annotateOldForWecom(
|
||||
oldJson, oldHighlight, typeGray, wrappersRemoved) + "”";
|
||||
String newRendered = newJson.isEmpty()
|
||||
? "" : "“" + SkeletonAnnotator.annotateNewForWecom(newJson, newHighlight) + "”";
|
||||
? "" : "“" + SkeletonAnnotator.annotateNewForWecom(
|
||||
newJson, newGreen, typeGray, wrappersAdded) + "”";
|
||||
if (oldJson.isEmpty() && !newJson.isEmpty()) {
|
||||
sb.append(" > **value 新增为:** ").append(newRendered).append("\n\n");
|
||||
sb.append(" > **value 新增为:** ").append(newRendered).append('\n');
|
||||
} else if (!oldJson.isEmpty() && newJson.isEmpty()) {
|
||||
sb.append(" > **value 原结构:** ").append(oldRendered).append("(已删除写入)\n\n");
|
||||
sb.append(" > **value 原结构:** ").append(oldRendered)
|
||||
.append(kc.isMq() ? "(已删除投递)\n" : "(已删除写入)\n");
|
||||
} else {
|
||||
sb.append(" > **value值由:** ").append(oldRendered).append('\n');
|
||||
sb.append(" > **变更为:** ").append(newRendered).append("\n\n");
|
||||
sb.append(" > **变更为:** ").append(newRendered).append('\n');
|
||||
}
|
||||
appendTypeChangeSummary(sb, kc.getFieldDetails());
|
||||
sb.append('\n');
|
||||
return sb.toString();
|
||||
}
|
||||
|
||||
/**
|
||||
* Key 行:展示解析后的模式或未解析时的源码表达式;未解析时追加灰色提示。
|
||||
* 反引号仅包裹 key 文本,避免与加粗/颜色嵌套冲突。
|
||||
* 类型变更单独一行摘要,避免仅靠骨架颜色/占位难以识别。
|
||||
* 例:{@code > **类型变更**: amount <font color="warning">BigDecimal → Integer</font>}
|
||||
*/
|
||||
private void appendTypeChangeSummary(StringBuilder sb, List<SchemaChange> details) {
|
||||
if (details == null || details.isEmpty()) {
|
||||
return;
|
||||
}
|
||||
List<String> parts = new ArrayList<>();
|
||||
Set<String> seen = new LinkedHashSet<>();
|
||||
for (SchemaChange c : details) {
|
||||
if (c == null || c.getChangeType() != ChangeType.TYPE_CHANGED) {
|
||||
continue;
|
||||
}
|
||||
String path = nvl(c.getFieldPath());
|
||||
if (path.isEmpty()) {
|
||||
continue;
|
||||
}
|
||||
String oldType = displayJavaType(c.getOldValue());
|
||||
String newType = displayJavaType(c.getNewValue());
|
||||
String dedup = path + "|" + oldType + "|" + newType;
|
||||
if (!seen.add(dedup)) {
|
||||
continue;
|
||||
}
|
||||
StringBuilder part = new StringBuilder();
|
||||
// 字段名用普通文本(避免反引号被企微渲染成色块)
|
||||
part.append(path);
|
||||
if (!oldType.isEmpty() || !newType.isEmpty()) {
|
||||
part.append(" <font color=\"warning\">")
|
||||
.append(oldType.isEmpty() ? "?" : oldType)
|
||||
.append(" → ")
|
||||
.append(newType.isEmpty() ? "?" : newType)
|
||||
.append("</font>");
|
||||
}
|
||||
parts.add(part.toString());
|
||||
}
|
||||
if (parts.isEmpty()) {
|
||||
return;
|
||||
}
|
||||
sb.append(" > **类型变更**: ");
|
||||
for (int i = 0; i < parts.size(); i++) {
|
||||
if (i > 0) {
|
||||
sb.append(";");
|
||||
}
|
||||
sb.append(parts.get(i));
|
||||
}
|
||||
sb.append('\n');
|
||||
}
|
||||
|
||||
/** 将 {@code BigDecimal/NUMBER} 转为展示用 {@code BigDecimal}。 */
|
||||
private static String displayJavaType(String typeLabel) {
|
||||
if (typeLabel == null) {
|
||||
return "";
|
||||
}
|
||||
String s = typeLabel.trim();
|
||||
if (s.isEmpty()) {
|
||||
return "";
|
||||
}
|
||||
int slash = s.indexOf('/');
|
||||
if (slash > 0) {
|
||||
s = s.substring(0, slash).trim();
|
||||
}
|
||||
return s;
|
||||
}
|
||||
|
||||
/**
|
||||
* Key / Topic 行:展示解析后的模式或未解析时的源码表达式;未解析时追加灰色提示。
|
||||
*/
|
||||
private void appendDestinationLine(StringBuilder sb, KeyStructureChange kc) {
|
||||
boolean mq = kc.isMq();
|
||||
String label = mq ? "Topic -->" : "Key -->";
|
||||
String keyText = kc.isKeyUnresolved() && kc.getKeyExpression() != null
|
||||
&& !kc.getKeyExpression().trim().isEmpty()
|
||||
? kc.getKeyExpression().trim()
|
||||
: nvl(kc.displayKey());
|
||||
if (keyText.isEmpty()) {
|
||||
keyText = "unknown-key";
|
||||
}
|
||||
sb.append("- ").append(label).append(" `").append(escapeWeComCode(keyText)).append('`');
|
||||
if (kc.isKeyUnresolved()) {
|
||||
sb.append(mq
|
||||
? " <font color=\"comment\">(destination 未解析)</font>"
|
||||
: " <font color=\"comment\">(key 无法解析)</font>");
|
||||
}
|
||||
sb.append('\n');
|
||||
}
|
||||
|
||||
/**
|
||||
* Key 行(无 KeyStructureChange 时的退路)。
|
||||
*/
|
||||
private void appendKeyLine(StringBuilder sb, String displayKey, String keyExpression,
|
||||
boolean unresolved) {
|
||||
@@ -156,17 +255,28 @@ public class ReportBuilder {
|
||||
if (keyText.isEmpty()) {
|
||||
keyText = "unknown-key";
|
||||
}
|
||||
sb.append("- Key --> `").append(keyText).append('`');
|
||||
sb.append("- Key --> `").append(escapeWeComCode(keyText)).append('`');
|
||||
if (unresolved) {
|
||||
sb.append(" <font color=\"comment\">(key 无法解析)</font>");
|
||||
}
|
||||
sb.append('\n');
|
||||
}
|
||||
|
||||
/** 位置、类型作为每个 Key 块的通用项。 */
|
||||
/**
|
||||
* 企微 markdown 代码片段内的转义:{@code *} 会触发斜体(即使包在反引号里)。
|
||||
*/
|
||||
static String escapeWeComCode(String text) {
|
||||
if (text == null || text.isEmpty()) {
|
||||
return "";
|
||||
}
|
||||
// 反斜杠转义对企微不可靠;用全角 * 保留通配语义且不被吃掉
|
||||
return text.replace("*", "*");
|
||||
}
|
||||
|
||||
/** 位置、类型作为每个 Key 块的通用项(不加反引号,避免企微渲染成色块)。 */
|
||||
private void appendMetaLines(StringBuilder sb, String writeLocation, String valueType) {
|
||||
sb.append(" > **位置**: `").append(nvl(writeLocation)).append("`\n");
|
||||
sb.append(" > **类型**: `").append(nvl(valueType)).append("`\n");
|
||||
sb.append(" > **位置**: ").append(nvl(writeLocation)).append('\n');
|
||||
sb.append(" > **类型**: ").append(nvl(valueType)).append('\n');
|
||||
}
|
||||
|
||||
private boolean isUnresolvedKey(String keyPattern) {
|
||||
@@ -202,7 +312,7 @@ public class ReportBuilder {
|
||||
for (SchemaChange c : list) {
|
||||
sb.append("- **").append(c.getChangeType().getLabel()).append("**");
|
||||
if (c.getKeyPattern() != null) {
|
||||
sb.append(" `").append(c.getKeyPattern()).append('`');
|
||||
sb.append(" `").append(escapeWeComCode(c.getKeyPattern())).append('`');
|
||||
}
|
||||
sb.append('\n');
|
||||
if (c.getWriteLocation() != null) {
|
||||
@@ -250,8 +360,8 @@ public class ReportBuilder {
|
||||
+ trimmed.substring(colonEn + 1).trim();
|
||||
}
|
||||
String[] knownPrefixes = {
|
||||
"删除字段 ", "新增字段 ", "字段路径迁移 ", "新增包装层 ", "新增缓存写入点,",
|
||||
"删除缓存写入点,原 value 类型: "
|
||||
"删除字段 ", "新增字段 ", "字段路径迁移 ", "新增包装层 ", "删除包装层 ",
|
||||
"新增缓存写入点,", "删除缓存写入点,原 value 类型: "
|
||||
};
|
||||
for (String prefix : knownPrefixes) {
|
||||
if (trimmed.startsWith(prefix)) {
|
||||
|
||||
@@ -12,23 +12,28 @@ import java.util.Set;
|
||||
/**
|
||||
* 在骨架 JSON 中为改动字段加企微颜色标注(仅改动片段染色,其余明文)。
|
||||
* <ul>
|
||||
* <li>删除字段 → 旧骨架,橙色 {@code warning}</li>
|
||||
* <li>新增字段 / 新增包装层 → 新骨架,绿色 {@code info}</li>
|
||||
* <li>路径迁移 → 旧路径橙、新路径绿</li>
|
||||
* <li>删除包装层 → 旧骨架优先整段标橙(先于子路径)</li>
|
||||
* <li>删除字段 / 路径迁移旧侧 → 旧骨架,橙色 {@code warning}</li>
|
||||
* <li>新增包装层 → 新骨架优先整体标绿(先于子路径)</li>
|
||||
* <li>新增字段 / 路径迁移新侧 → 新骨架,绿色 {@code info}</li>
|
||||
* <li>类型变更(修改类)→ 新旧骨架均为灰色 {@code comment}</li>
|
||||
* </ul>
|
||||
*/
|
||||
final class SkeletonAnnotator {
|
||||
|
||||
/** 企微橙:删除 / 旧侧变更 */
|
||||
/** 企微橙:删除 / 路径迁移旧侧 / 删除包装层 */
|
||||
static final String COLOR_REMOVE = "warning";
|
||||
/** 企微绿:新增 / 新侧变更 */
|
||||
/** 企微绿:新增 / 新侧路径迁移 */
|
||||
static final String COLOR_ADD = "info";
|
||||
/** 企微灰:类型变更(修改类) */
|
||||
static final String COLOR_TYPE = "comment";
|
||||
|
||||
private static final String FONT_CLOSE = "</font>";
|
||||
|
||||
private SkeletonAnnotator() {
|
||||
}
|
||||
|
||||
/** 旧骨架应橙色标注的路径:删除、路径迁移旧侧(不含类型变更与包装层删除)。 */
|
||||
static Set<String> pathsForOldSkeleton(List<SchemaChange> details) {
|
||||
Set<String> paths = new LinkedHashSet<>();
|
||||
if (details == null) {
|
||||
@@ -47,6 +52,35 @@ final class SkeletonAnnotator {
|
||||
return paths;
|
||||
}
|
||||
|
||||
/** 删除包装层路径(旧骨架优先整段标橙)。 */
|
||||
static Set<String> pathsForWrapperRemoved(List<SchemaChange> details) {
|
||||
Set<String> paths = new LinkedHashSet<>();
|
||||
if (details == null) {
|
||||
return paths;
|
||||
}
|
||||
for (SchemaChange c : details) {
|
||||
if (c != null && c.getChangeType() == ChangeType.WRAPPER_REMOVED) {
|
||||
addIfPresent(paths, c.getFieldPath());
|
||||
}
|
||||
}
|
||||
return paths;
|
||||
}
|
||||
|
||||
/** 新增包装层路径(新骨架优先整段标绿)。 */
|
||||
static Set<String> pathsForWrapperAdded(List<SchemaChange> details) {
|
||||
Set<String> paths = new LinkedHashSet<>();
|
||||
if (details == null) {
|
||||
return paths;
|
||||
}
|
||||
for (SchemaChange c : details) {
|
||||
if (c != null && c.getChangeType() == ChangeType.WRAPPER_ADDED) {
|
||||
addIfPresent(paths, c.getFieldPath());
|
||||
}
|
||||
}
|
||||
return paths;
|
||||
}
|
||||
|
||||
/** 新骨架应绿色标注的路径:新增字段、路径迁移新侧(不含包装层与类型变更)。 */
|
||||
static Set<String> pathsForNewSkeleton(List<SchemaChange> details) {
|
||||
Set<String> paths = new LinkedHashSet<>();
|
||||
if (details == null) {
|
||||
@@ -56,8 +90,7 @@ final class SkeletonAnnotator {
|
||||
if (c == null || c.getChangeType() == null) {
|
||||
continue;
|
||||
}
|
||||
if (c.getChangeType() == ChangeType.FIELD_ADDED
|
||||
|| c.getChangeType() == ChangeType.WRAPPER_ADDED) {
|
||||
if (c.getChangeType() == ChangeType.FIELD_ADDED) {
|
||||
addIfPresent(paths, c.getFieldPath());
|
||||
} else if (c.getChangeType() == ChangeType.FIELD_PATH_MOVED) {
|
||||
addIfPresent(paths, c.getFieldPath());
|
||||
@@ -69,23 +102,68 @@ final class SkeletonAnnotator {
|
||||
return paths;
|
||||
}
|
||||
|
||||
/**
|
||||
* 标注旧骨架改动字段(删除 → 橙色)。
|
||||
*/
|
||||
static String annotateOldForWecom(String json, Set<String> highlightPaths) {
|
||||
return annotateForWecom(json, highlightPaths, COLOR_REMOVE);
|
||||
/** 类型变更路径:新旧骨架均用灰色标注。 */
|
||||
static Set<String> pathsForTypeChanged(List<SchemaChange> details) {
|
||||
Set<String> paths = new LinkedHashSet<>();
|
||||
if (details == null) {
|
||||
return paths;
|
||||
}
|
||||
for (SchemaChange c : details) {
|
||||
if (c != null && c.getChangeType() == ChangeType.TYPE_CHANGED) {
|
||||
addIfPresent(paths, c.getFieldPath());
|
||||
}
|
||||
}
|
||||
return paths;
|
||||
}
|
||||
|
||||
/**
|
||||
* 标注新骨架改动字段(新增 → 绿色)。
|
||||
* 标注旧骨架:删除等橙色;类型变更灰色。
|
||||
*/
|
||||
static String annotateNewForWecom(String json, Set<String> highlightPaths) {
|
||||
return annotateForWecom(json, highlightPaths, COLOR_ADD);
|
||||
static String annotateOldForWecom(String json, Set<String> orangePaths) {
|
||||
return annotateOldForWecom(json, orangePaths, null, null);
|
||||
}
|
||||
|
||||
static String annotateOldForWecom(String json, Set<String> orangePaths, Set<String> grayPaths) {
|
||||
return annotateOldForWecom(json, orangePaths, grayPaths, null);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param wrapperRemovedPaths 删除的包装层,优先于普通橙路径标注
|
||||
*/
|
||||
static String annotateOldForWecom(String json, Set<String> orangePaths,
|
||||
Set<String> grayPaths, Set<String> wrapperRemovedPaths) {
|
||||
String result = annotateForWecom(json, grayPaths, COLOR_TYPE);
|
||||
result = annotateForWecom(result, wrapperRemovedPaths, COLOR_REMOVE);
|
||||
return annotateForWecom(result, orangePaths, COLOR_REMOVE);
|
||||
}
|
||||
|
||||
/**
|
||||
* 标注新骨架:类型变更灰色,新增绿色。
|
||||
*/
|
||||
static String annotateNewForWecom(String json, Set<String> greenPaths) {
|
||||
return annotateNewForWecom(json, greenPaths, null, null);
|
||||
}
|
||||
|
||||
/**
|
||||
* 标注新骨架(类型变更 → 灰色;包装层先绿;其余新增/迁移 → 绿)。
|
||||
*/
|
||||
static String annotateNewForWecom(String json, Set<String> greenPaths, Set<String> grayTypePaths) {
|
||||
return annotateNewForWecom(json, greenPaths, grayTypePaths, null);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param wrapperPaths 包装层路径,优先于 {@code greenPaths} 标注,使 {@code "vo":{...}} 能整段染色
|
||||
*/
|
||||
static String annotateNewForWecom(String json, Set<String> greenPaths,
|
||||
Set<String> grayTypePaths, Set<String> wrapperPaths) {
|
||||
String result = annotateForWecom(json, grayTypePaths, COLOR_TYPE);
|
||||
result = annotateForWecom(result, wrapperPaths, COLOR_ADD);
|
||||
return annotateForWecom(result, greenPaths, COLOR_ADD);
|
||||
}
|
||||
|
||||
/**
|
||||
* 仅标注改动字段:其余正文保持普通文本。
|
||||
* 改动片段格式:{@code <font color="warning|info">"field":value</font>}
|
||||
* 改动片段格式:{@code <font color="warning|info|comment">"field":value</font>}
|
||||
*/
|
||||
static String annotateForWecom(String json, Set<String> highlightPaths) {
|
||||
return annotateForWecom(json, highlightPaths, COLOR_REMOVE);
|
||||
|
||||
@@ -21,12 +21,11 @@ public class JavaSchemaExtractor {
|
||||
|
||||
private static final Set<String> STRING_TYPES = new HashSet<>(Arrays.asList(
|
||||
"String", "CharSequence", "char", "Character", "UUID",
|
||||
"Date", "LocalDate", "LocalDateTime", "LocalTime", "Instant", "Timestamp",
|
||||
"BigDecimal"));
|
||||
"Date", "LocalDate", "LocalDateTime", "LocalTime", "Instant", "Timestamp"));
|
||||
private static final Set<String> NUMBER_TYPES = new HashSet<>(Arrays.asList(
|
||||
"int", "long", "short", "byte", "double", "float",
|
||||
"Integer", "Long", "Short", "Byte", "Double", "Float",
|
||||
"Number", "BigInteger", "AtomicInteger", "AtomicLong"));
|
||||
"Number", "BigInteger", "BigDecimal", "AtomicInteger", "AtomicLong"));
|
||||
private static final Set<String> BOOLEAN_TYPES = new HashSet<>(Arrays.asList(
|
||||
"boolean", "Boolean"));
|
||||
private static final Set<String> COLLECTION_TYPES = new HashSet<>(Arrays.asList(
|
||||
|
||||
@@ -45,7 +45,7 @@ public class SkeletonJsonRenderer {
|
||||
private Node buildTree(TypeSchema schema) {
|
||||
Node root = new Node(JsonType.OBJECT);
|
||||
for (FieldSchema field : schema.getFields().values()) {
|
||||
putPath(root, field.getPath(), field.getJsonType());
|
||||
putPath(root, field.getPath(), field.getJsonType(), field.getJavaType());
|
||||
}
|
||||
// 根数组:字段以 [] / [].xxx 记录
|
||||
if (root.children.size() == 1 && root.children.containsKey("[]")) {
|
||||
@@ -56,7 +56,7 @@ public class SkeletonJsonRenderer {
|
||||
return root;
|
||||
}
|
||||
|
||||
private void putPath(Node root, String path, JsonType type) {
|
||||
private void putPath(Node root, String path, JsonType type, String javaType) {
|
||||
List<Seg> segs = parsePath(path);
|
||||
if (segs.isEmpty()) {
|
||||
return;
|
||||
@@ -70,10 +70,9 @@ public class SkeletonJsonRenderer {
|
||||
arr.type = JsonType.ARRAY;
|
||||
Node elem = arr.children.computeIfAbsent("[]", k -> new Node(JsonType.OBJECT));
|
||||
if (last) {
|
||||
if (type == JsonType.OBJECT || type == JsonType.ARRAY || type == JsonType.MAP) {
|
||||
elem.type = type;
|
||||
} else {
|
||||
elem.type = type;
|
||||
elem.javaType = javaType;
|
||||
if (type != JsonType.OBJECT && type != JsonType.ARRAY && type != JsonType.MAP) {
|
||||
elem.leaf = true;
|
||||
}
|
||||
}
|
||||
@@ -83,6 +82,7 @@ public class SkeletonJsonRenderer {
|
||||
k -> new Node(last ? type : JsonType.OBJECT));
|
||||
if (last) {
|
||||
child.type = type;
|
||||
child.javaType = javaType;
|
||||
child.leaf = type != JsonType.OBJECT && type != JsonType.ARRAY && type != JsonType.MAP;
|
||||
} else if (child.type != JsonType.ARRAY) {
|
||||
child.type = JsonType.OBJECT;
|
||||
@@ -129,7 +129,7 @@ public class SkeletonJsonRenderer {
|
||||
return "[" + write(elem, elemPath, protectedPaths, compact) + "]";
|
||||
}
|
||||
if (node.leaf || isScalar(node.type)) {
|
||||
return placeholder(node.type);
|
||||
return placeholder(node.type, node.javaType);
|
||||
}
|
||||
if (node.type == JsonType.MAP) {
|
||||
return "{}";
|
||||
@@ -161,7 +161,7 @@ public class SkeletonJsonRenderer {
|
||||
} else if (child.type == JsonType.ARRAY) {
|
||||
sb.append(write(child, childPath, protectedPaths, compact));
|
||||
} else if (child.leaf || isScalar(child.type)) {
|
||||
sb.append(placeholder(child.type));
|
||||
sb.append(placeholder(child.type, child.javaType));
|
||||
} else {
|
||||
sb.append(write(child, childPath, protectedPaths, compact));
|
||||
}
|
||||
@@ -177,7 +177,7 @@ public class SkeletonJsonRenderer {
|
||||
if (child.type == JsonType.OBJECT || child.type == JsonType.MAP) {
|
||||
return "\"...\"";
|
||||
}
|
||||
return placeholder(child.type);
|
||||
return placeholder(child.type, child.javaType);
|
||||
}
|
||||
|
||||
private boolean isProtectedUnder(String pathPrefix, Set<String> protectedPaths) {
|
||||
@@ -327,13 +327,17 @@ public class SkeletonJsonRenderer {
|
||||
|| type == JsonType.BOOLEAN || type == JsonType.UNKNOWN;
|
||||
}
|
||||
|
||||
private String placeholder(JsonType type) {
|
||||
/**
|
||||
* 数字类型示意占位(合法 JSON number):整数 {@code 0}、浮点 {@code 0.0}、BigDecimal {@code 0.00}。
|
||||
* 仅用于通知可读性,不代表真实精度/scale。
|
||||
*/
|
||||
private String placeholder(JsonType type, String javaType) {
|
||||
if (type == null) {
|
||||
return "null";
|
||||
}
|
||||
switch (type) {
|
||||
case NUMBER:
|
||||
return "0";
|
||||
return numberPlaceholder(javaType);
|
||||
case BOOLEAN:
|
||||
return "false";
|
||||
case STRING:
|
||||
@@ -349,6 +353,37 @@ public class SkeletonJsonRenderer {
|
||||
}
|
||||
}
|
||||
|
||||
private String numberPlaceholder(String javaType) {
|
||||
String simple = simpleJavaType(javaType);
|
||||
if ("BigDecimal".equals(simple)) {
|
||||
return "0.00";
|
||||
}
|
||||
if ("float".equals(simple) || "Float".equals(simple)
|
||||
|| "double".equals(simple) || "Double".equals(simple)) {
|
||||
return "0.0";
|
||||
}
|
||||
return "0";
|
||||
}
|
||||
|
||||
private static String simpleJavaType(String javaType) {
|
||||
if (javaType == null) {
|
||||
return "";
|
||||
}
|
||||
String s = javaType.trim();
|
||||
if (s.isEmpty()) {
|
||||
return "";
|
||||
}
|
||||
int lt = s.indexOf('<');
|
||||
if (lt > 0) {
|
||||
s = s.substring(0, lt).trim();
|
||||
}
|
||||
int dot = s.lastIndexOf('.');
|
||||
if (dot >= 0 && dot < s.length() - 1) {
|
||||
s = s.substring(dot + 1);
|
||||
}
|
||||
return s;
|
||||
}
|
||||
|
||||
private String escape(String s) {
|
||||
return s.replace("\\", "\\\\").replace("\"", "\\\"");
|
||||
}
|
||||
@@ -365,6 +400,7 @@ public class SkeletonJsonRenderer {
|
||||
|
||||
private static final class Node {
|
||||
JsonType type;
|
||||
String javaType;
|
||||
boolean leaf;
|
||||
final Map<String, Node> children = new LinkedHashMap<>();
|
||||
|
||||
|
||||
@@ -35,6 +35,8 @@ ignore:
|
||||
- "**/test/**"
|
||||
# 忽略的写入方法(类全名#方法名)
|
||||
writer_methods: []
|
||||
# 忽略的 MQ destination(topic 或 topic:tag,glob)
|
||||
mq_destinations: []
|
||||
|
||||
# 检测规则
|
||||
detection:
|
||||
@@ -44,8 +46,21 @@ detection:
|
||||
- W03 # stringRedisTemplate.opsForValue().set(key, JsonUtil.getObjectToString(x), ...)
|
||||
- W04 # redisTemplate.opsForValue().set(key, obj, ...)
|
||||
- W05 # redisTemplate.opsForHash().put(key, field, obj)
|
||||
# MQ 生产侧投递检测(Phase M1 + M2)
|
||||
mq_patterns:
|
||||
- MQ01 # rocketMQTemplate.syncSend(dest, payload)
|
||||
- MQ02 # asyncSend / syncSendOrderly / sendOneWay
|
||||
- MQ03 # convertAndSend(dest, payload)
|
||||
- MQ04 # MessageBuilder.withPayload / Message<T>
|
||||
- MQ05 # 先 toJSONString 再 send String
|
||||
- MQ-K01 # kafkaTemplate.send(topic, payload)
|
||||
- MQ-K02 # kafkaTemplate.send(topic, key, payload)
|
||||
- MQ-K03 # kafkaTemplate.send(ProducerRecord)
|
||||
- MQ-K04 # 先 JSON 序列化为 String 再 send
|
||||
# W06 读侧辅助:用 parseObject / getJsonToBean 等补强写入点 value 类型(非写入模式)
|
||||
read_hints_enabled: true
|
||||
# MQ-R 读侧补强(Listener / parseObject)
|
||||
mq_read_hints_enabled: true
|
||||
# 类型推断最低置信度,低于此值标记为低置信度提示
|
||||
min_confidence: 0.6
|
||||
# 字段展开最大深度(防止循环引用)
|
||||
|
||||
185
src/test/java/com/codechecker/cache/MqScenarioTest.java
vendored
Normal file
185
src/test/java/com/codechecker/cache/MqScenarioTest.java
vendored
Normal file
@@ -0,0 +1,185 @@
|
||||
package com.codechecker.cache;
|
||||
|
||||
import com.codechecker.cache.detector.CacheReadHint;
|
||||
import com.codechecker.cache.detector.MqReadHintDetector;
|
||||
import com.codechecker.cache.detector.MqWritePointDetector;
|
||||
import com.codechecker.cache.detector.WritePoint;
|
||||
import com.codechecker.cache.diff.ChangeType;
|
||||
import com.codechecker.cache.diff.SchemaChange;
|
||||
import com.codechecker.cache.diff.SchemaDiffer;
|
||||
import com.codechecker.cache.report.CheckReport;
|
||||
import com.codechecker.cache.report.KeyStructureChange;
|
||||
import com.codechecker.cache.report.ReportBuilder;
|
||||
import com.codechecker.cache.schema.JavaSchemaExtractor;
|
||||
import com.codechecker.cache.schema.SourceIndex;
|
||||
import com.codechecker.cache.schema.TypeSchema;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
import java.util.Set;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertFalse;
|
||||
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||
|
||||
class MqScenarioTest {
|
||||
|
||||
@Test
|
||||
void rocketMqFieldRemovalProducesTopicDiff() {
|
||||
String constants = TestSupport.fixture("fixtures/mq/rocket-wallet/CapitalMqConstants.txt");
|
||||
String oldReq = TestSupport.fixture("fixtures/mq/rocket-wallet/WalletDeductReqOld.txt");
|
||||
String newReq = TestSupport.fixture("fixtures/mq/rocket-wallet/WalletDeductReqNew.txt");
|
||||
String producer = TestSupport.fixture("fixtures/mq/rocket-wallet/WalletDeductProducer.txt");
|
||||
|
||||
SourceIndex oldIndex = new SourceIndex();
|
||||
oldIndex.addSource(constants);
|
||||
oldIndex.addSource(oldReq);
|
||||
oldIndex.addSource(producer);
|
||||
|
||||
SourceIndex newIndex = new SourceIndex();
|
||||
newIndex.addSource(constants);
|
||||
newIndex.addSource(newReq);
|
||||
newIndex.addSource(producer);
|
||||
|
||||
Set<String> patterns = new HashSet<>();
|
||||
patterns.add("MQ01");
|
||||
WritePoint wp = new MqWritePointDetector(newIndex, patterns)
|
||||
.detect("WalletDeductProducer.java", producer).get(0);
|
||||
|
||||
TypeSchema oldSchema = new JavaSchemaExtractor(oldIndex, 8)
|
||||
.extract(wp.getResolvedValueType(), wp.isRootArray());
|
||||
TypeSchema newSchema = new JavaSchemaExtractor(newIndex, 8)
|
||||
.extract(wp.getResolvedValueType(), wp.isRootArray());
|
||||
List<SchemaChange> changes = new SchemaDiffer().diff(oldSchema, newSchema);
|
||||
List<ChangeType> types = changes.stream().map(SchemaChange::getChangeType).collect(Collectors.toList());
|
||||
assertTrue(types.contains(ChangeType.FIELD_REMOVED), "应删除 remark 字段");
|
||||
|
||||
CheckReport report = new CheckReport();
|
||||
report.setRepository("demo");
|
||||
report.setBranch("feature/mq");
|
||||
report.setOldSha("aaa");
|
||||
report.setNewSha("bbb");
|
||||
KeyStructureChange key = new KeyStructureChange();
|
||||
key.setKeyPattern(wp.getResolvedKeyPattern());
|
||||
key.setChannel(WritePoint.CHANNEL_ROCKETMQ);
|
||||
key.setWriteLocation(wp.location());
|
||||
key.setValueType("WalletDeductReq");
|
||||
key.setOldSkeletonJson("{\"walletId\":\"\",\"amount\":0,\"remark\":\"\"}");
|
||||
key.setNewSkeletonJson("{\"walletId\":\"\",\"amount\":0}");
|
||||
key.getFieldDetails().addAll(changes);
|
||||
report.getKeyChanges().add(key);
|
||||
report.getChanges().addAll(changes);
|
||||
|
||||
String md = new ReportBuilder("[序列化结构变更]").toMarkdown(report);
|
||||
assertTrue(md.contains("- Topic --> `capital-topic:WALLET_DEDUCT`"), md);
|
||||
assertTrue(md.contains("> **通道**: RocketMQ"), md);
|
||||
assertTrue(md.contains("<font color=\"warning\">\"remark\":\"\"</font>")
|
||||
|| md.contains("remark"), md);
|
||||
}
|
||||
|
||||
@Test
|
||||
void kafkaListFieldRemovalProducesTopicDiff() {
|
||||
String oldVo = TestSupport.fixture("fixtures/mq/kafka-patrol/CheckItemDetailVoOld.txt");
|
||||
String newVo = TestSupport.fixture("fixtures/mq/kafka-patrol/CheckItemDetailVoNew.txt");
|
||||
String service = TestSupport.fixture("fixtures/mq/kafka-patrol/PatrolService.txt");
|
||||
|
||||
SourceIndex oldIndex = new SourceIndex();
|
||||
oldIndex.addSource(oldVo);
|
||||
oldIndex.addSource(service);
|
||||
SourceIndex newIndex = new SourceIndex();
|
||||
newIndex.addSource(newVo);
|
||||
newIndex.addSource(service);
|
||||
|
||||
Set<String> patterns = new HashSet<>();
|
||||
patterns.add("MQ-K01");
|
||||
WritePoint wp = new MqWritePointDetector(newIndex, patterns)
|
||||
.detect("PatrolService.java", service).get(0);
|
||||
assertEquals("patrol-store-food-safe:*", wp.getResolvedKeyPattern());
|
||||
assertTrue(wp.isRootArray());
|
||||
|
||||
TypeSchema oldSchema = new JavaSchemaExtractor(oldIndex, 8)
|
||||
.extract(wp.getResolvedValueType(), true);
|
||||
TypeSchema newSchema = new JavaSchemaExtractor(newIndex, 8)
|
||||
.extract(wp.getResolvedValueType(), true);
|
||||
List<SchemaChange> changes = new SchemaDiffer().diff(oldSchema, newSchema);
|
||||
assertTrue(changes.stream().anyMatch(c -> c.getChangeType() == ChangeType.FIELD_REMOVED));
|
||||
|
||||
CheckReport report = new CheckReport();
|
||||
report.setRepository("demo");
|
||||
KeyStructureChange key = new KeyStructureChange();
|
||||
key.setKeyPattern(wp.getResolvedKeyPattern());
|
||||
key.setChannel(WritePoint.CHANNEL_KAFKA);
|
||||
key.setValueType("List<CheckItemDetailVo>");
|
||||
key.setWriteLocation(wp.location());
|
||||
key.setOldSkeletonJson("[{\"itemId\":\"\",\"itemName\":\"\",\"score\":0}]");
|
||||
key.setNewSkeletonJson("[{\"itemId\":\"\",\"itemName\":\"\"}]");
|
||||
key.getFieldDetails().addAll(changes);
|
||||
report.getKeyChanges().add(key);
|
||||
|
||||
String md = new ReportBuilder("[序列化结构变更]").toMarkdown(report);
|
||||
assertTrue(md.contains("- Topic --> `patrol-store-food-safe:*`"), md);
|
||||
assertTrue(md.contains("> **通道**: Kafka"), md);
|
||||
assertTrue(md.contains("> **类型**: List<CheckItemDetailVo>"), md);
|
||||
}
|
||||
|
||||
@Test
|
||||
void messageBuilderProducerResolvesDutyImNotice() {
|
||||
String notice = TestSupport.fixture("fixtures/mq/rocket-im/DutyImNotice.txt");
|
||||
String producer = TestSupport.fixture("fixtures/mq/rocket-im/DutyImNoticeProducer.txt");
|
||||
|
||||
SourceIndex index = new SourceIndex();
|
||||
index.addSource(notice);
|
||||
index.addSource(producer);
|
||||
|
||||
Set<String> patterns = new HashSet<>();
|
||||
patterns.add("MQ02");
|
||||
patterns.add("MQ04");
|
||||
WritePoint wp = new MqWritePointDetector(index, patterns)
|
||||
.detect("DutyImNoticeProducer.java", producer).stream()
|
||||
.filter(w -> "MQ04".equals(w.getPattern()))
|
||||
.findFirst()
|
||||
.orElseThrow(() -> new AssertionError("应命中 MQ04"));
|
||||
assertEquals("duty-im-notice-topic", wp.getResolvedKeyPattern());
|
||||
assertTrue(wp.getResolvedValueType().endsWith("DutyImNotice"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void mqReadHintEnrichesWritePointByDestination() {
|
||||
String notice = TestSupport.fixture("fixtures/mq/rocket-im/DutyImNotice.txt");
|
||||
String producer = TestSupport.fixture("fixtures/mq/rocket-im/DutyImNoticeProducer.txt");
|
||||
String consumer = TestSupport.fixture("fixtures/mq/rocket-im/DutyImNoticeConsumer.txt");
|
||||
|
||||
SourceIndex index = new SourceIndex();
|
||||
index.addSource(notice);
|
||||
index.addSource(producer);
|
||||
index.addSource(consumer);
|
||||
|
||||
List<CacheReadHint> hints = new MqReadHintDetector(index)
|
||||
.detect("DutyImNoticeConsumer.java", consumer);
|
||||
assertFalse(hints.isEmpty());
|
||||
|
||||
Set<String> patterns = new HashSet<>();
|
||||
patterns.add("MQ02");
|
||||
patterns.add("MQ04");
|
||||
WritePoint wp = new MqWritePointDetector(index, patterns)
|
||||
.detect("DutyImNoticeProducer.java", producer).stream()
|
||||
.filter(w -> "MQ04".equals(w.getPattern()))
|
||||
.findFirst()
|
||||
.orElseThrow(() -> new AssertionError("应命中 MQ04"));
|
||||
|
||||
// 模拟无泛型 Message 导致类型缺失时,MQ-R 可按 destination 补强
|
||||
wp.setResolvedValueType(null);
|
||||
wp.setConfidence(0.4);
|
||||
CacheReadHint hint = hints.get(0);
|
||||
assertEquals(wp.getResolvedKeyPattern(), hint.getResolvedKeyPattern());
|
||||
|
||||
if (hint.getResolvedValueType() != null) {
|
||||
wp.setResolvedValueType(hint.getResolvedValueType());
|
||||
wp.setConfidence(Math.max(wp.getConfidence(), hint.getConfidence()));
|
||||
}
|
||||
assertTrue(wp.getResolvedValueType().endsWith("DutyImNotice"));
|
||||
assertTrue(wp.getConfidence() >= 0.8);
|
||||
}
|
||||
}
|
||||
@@ -19,6 +19,11 @@ class ConfigLoaderTest {
|
||||
assertTrue(config.isEnabled());
|
||||
assertEquals("notify", config.getMode());
|
||||
assertTrue(config.getDetection().getPatterns().contains("W01"));
|
||||
assertTrue(config.getDetection().getMqPatterns().contains("MQ01"));
|
||||
assertTrue(config.getDetection().getMqPatterns().contains("MQ-K01"));
|
||||
assertTrue(config.getDetection().getMqPatterns().contains("MQ04"));
|
||||
assertTrue(config.getDetection().getMqPatterns().contains("MQ-K03"));
|
||||
assertTrue(config.getDetection().isMqReadHintsEnabled());
|
||||
assertFalse(config.isScanTestSources());
|
||||
}
|
||||
|
||||
|
||||
51
src/test/java/com/codechecker/cache/detector/MqReadHintDetectorTest.java
vendored
Normal file
51
src/test/java/com/codechecker/cache/detector/MqReadHintDetectorTest.java
vendored
Normal file
@@ -0,0 +1,51 @@
|
||||
package com.codechecker.cache.detector;
|
||||
|
||||
import com.codechecker.cache.TestSupport;
|
||||
import com.codechecker.cache.schema.SourceIndex;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertFalse;
|
||||
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||
|
||||
class MqReadHintDetectorTest {
|
||||
|
||||
@Test
|
||||
void detectsRocketMqListenerGenericType() {
|
||||
String notice = TestSupport.fixture("fixtures/mq/rocket-im/DutyImNotice.txt");
|
||||
String producer = TestSupport.fixture("fixtures/mq/rocket-im/DutyImNoticeProducer.txt");
|
||||
String consumer = TestSupport.fixture("fixtures/mq/rocket-im/DutyImNoticeConsumer.txt");
|
||||
|
||||
SourceIndex index = new SourceIndex();
|
||||
index.addSource(notice);
|
||||
index.addSource(producer);
|
||||
index.addSource(consumer);
|
||||
|
||||
List<CacheReadHint> hints = new MqReadHintDetector(index)
|
||||
.detect("DutyImNoticeConsumer.java", consumer);
|
||||
assertFalse(hints.isEmpty());
|
||||
CacheReadHint hint = hints.get(0);
|
||||
assertTrue(hint.getResolvedValueType().endsWith("DutyImNotice"));
|
||||
assertEquals("duty-im-notice-topic", hint.getResolvedKeyPattern());
|
||||
assertFalse(hint.isRootArray());
|
||||
}
|
||||
|
||||
@Test
|
||||
void detectsKafkaListenerParseObject() {
|
||||
String vo = TestSupport.fixture("fixtures/mq/kafka-record/PatrolNotifyVo.txt");
|
||||
String listener = TestSupport.fixture("fixtures/mq/kafka-record/PatrolNotifyListener.txt");
|
||||
|
||||
SourceIndex index = new SourceIndex();
|
||||
index.addSource(vo);
|
||||
index.addSource(listener);
|
||||
|
||||
List<CacheReadHint> hints = new MqReadHintDetector(index)
|
||||
.detect("PatrolNotifyListener.java", listener);
|
||||
assertTrue(hints.stream().anyMatch(h ->
|
||||
h.getResolvedValueType() != null
|
||||
&& h.getResolvedValueType().endsWith("PatrolNotifyVo")
|
||||
&& "patrol-notify-topic".equals(h.getResolvedKeyPattern())));
|
||||
}
|
||||
}
|
||||
175
src/test/java/com/codechecker/cache/detector/MqWritePointDetectorTest.java
vendored
Normal file
175
src/test/java/com/codechecker/cache/detector/MqWritePointDetectorTest.java
vendored
Normal file
@@ -0,0 +1,175 @@
|
||||
package com.codechecker.cache.detector;
|
||||
|
||||
import com.codechecker.cache.TestSupport;
|
||||
import com.codechecker.cache.schema.SourceIndex;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
import java.util.Set;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertFalse;
|
||||
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||
|
||||
class MqWritePointDetectorTest {
|
||||
|
||||
@Test
|
||||
void detectsRocketMqSyncSendWithTopicTag() {
|
||||
String constants = TestSupport.fixture("fixtures/mq/rocket-wallet/CapitalMqConstants.txt");
|
||||
String req = TestSupport.fixture("fixtures/mq/rocket-wallet/WalletDeductReqOld.txt");
|
||||
String producer = TestSupport.fixture("fixtures/mq/rocket-wallet/WalletDeductProducer.txt");
|
||||
|
||||
SourceIndex index = new SourceIndex();
|
||||
index.addSource(constants);
|
||||
index.addSource(req);
|
||||
index.addSource(producer);
|
||||
|
||||
Set<String> patterns = new HashSet<>();
|
||||
patterns.add("MQ01");
|
||||
patterns.add("MQ02");
|
||||
patterns.add("MQ03");
|
||||
patterns.add("MQ04");
|
||||
patterns.add("MQ05");
|
||||
List<WritePoint> wps = new MqWritePointDetector(index, patterns)
|
||||
.detect("WalletDeductProducer.java", producer);
|
||||
|
||||
assertEquals(1, wps.size());
|
||||
WritePoint wp = wps.get(0);
|
||||
assertEquals("MQ01", wp.getPattern());
|
||||
assertEquals(WritePoint.CHANNEL_ROCKETMQ, wp.getChannel());
|
||||
assertEquals("capital-topic:WALLET_DEDUCT", wp.getResolvedKeyPattern());
|
||||
assertTrue(wp.getResolvedValueType().endsWith("WalletDeductReq"));
|
||||
assertFalse(wp.isRootArray());
|
||||
}
|
||||
|
||||
@Test
|
||||
void detectsKafkaSendListAsRootArray() {
|
||||
String vo = TestSupport.fixture("fixtures/mq/kafka-patrol/CheckItemDetailVoOld.txt");
|
||||
String service = TestSupport.fixture("fixtures/mq/kafka-patrol/PatrolService.txt");
|
||||
|
||||
SourceIndex index = new SourceIndex();
|
||||
index.addSource(vo);
|
||||
index.addSource(service);
|
||||
|
||||
Set<String> patterns = new HashSet<>();
|
||||
patterns.add("MQ-K01");
|
||||
patterns.add("MQ-K02");
|
||||
List<WritePoint> wps = new MqWritePointDetector(index, patterns)
|
||||
.detect("PatrolService.java", service);
|
||||
|
||||
assertEquals(1, wps.size());
|
||||
WritePoint wp = wps.get(0);
|
||||
assertEquals("MQ-K01", wp.getPattern());
|
||||
assertEquals(WritePoint.CHANNEL_KAFKA, wp.getChannel());
|
||||
assertEquals("patrol-store-food-safe:*", wp.getResolvedKeyPattern());
|
||||
assertTrue(wp.getResolvedValueType().endsWith("CheckItemDetailVo"));
|
||||
assertTrue(wp.isRootArray());
|
||||
}
|
||||
|
||||
@Test
|
||||
void detectsMessageBuilderAsyncSendAsMq04() {
|
||||
String notice = TestSupport.fixture("fixtures/mq/rocket-im/DutyImNotice.txt");
|
||||
String producer = TestSupport.fixture("fixtures/mq/rocket-im/DutyImNoticeProducer.txt");
|
||||
|
||||
SourceIndex index = new SourceIndex();
|
||||
index.addSource(notice);
|
||||
index.addSource(producer);
|
||||
|
||||
Set<String> patterns = allRocketPatterns();
|
||||
List<WritePoint> wps = new MqWritePointDetector(index, patterns)
|
||||
.detect("DutyImNoticeProducer.java", producer);
|
||||
|
||||
WritePoint mq04 = findByPattern(wps, "MQ04");
|
||||
assertEquals(WritePoint.CHANNEL_ROCKETMQ, mq04.getChannel());
|
||||
assertEquals("duty-im-notice-topic", mq04.getResolvedKeyPattern());
|
||||
assertTrue(mq04.getResolvedValueType().endsWith("DutyImNotice"), mq04.getResolvedValueType());
|
||||
}
|
||||
|
||||
@Test
|
||||
void detectsConvertAndSendAsMq03() {
|
||||
String notice = TestSupport.fixture("fixtures/mq/rocket-im/DutyImNotice.txt");
|
||||
String producer = TestSupport.fixture("fixtures/mq/rocket-im/DutyImNoticeProducer.txt");
|
||||
SourceIndex index = new SourceIndex();
|
||||
index.addSource(notice);
|
||||
index.addSource(producer);
|
||||
|
||||
WritePoint mq03 = findByPattern(
|
||||
new MqWritePointDetector(index, allRocketPatterns())
|
||||
.detect("DutyImNoticeProducer.java", producer),
|
||||
"MQ03");
|
||||
assertTrue(mq03.getResolvedValueType().endsWith("DutyImNotice"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void detectsJsonStringSyncSendAsMq05() {
|
||||
String notice = TestSupport.fixture("fixtures/mq/rocket-im/DutyImNotice.txt");
|
||||
String producer = TestSupport.fixture("fixtures/mq/rocket-im/DutyImNoticeProducer.txt");
|
||||
SourceIndex index = new SourceIndex();
|
||||
index.addSource(notice);
|
||||
index.addSource(producer);
|
||||
|
||||
WritePoint mq05 = findByPattern(
|
||||
new MqWritePointDetector(index, allRocketPatterns())
|
||||
.detect("DutyImNoticeProducer.java", producer),
|
||||
"MQ05");
|
||||
assertTrue(mq05.getResolvedValueType().endsWith("DutyImNotice"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void detectsProducerRecordSendAsMqK03() {
|
||||
String vo = TestSupport.fixture("fixtures/mq/kafka-record/PatrolNotifyVo.txt");
|
||||
String producer = TestSupport.fixture("fixtures/mq/kafka-record/PatrolNotifyProducer.txt");
|
||||
SourceIndex index = new SourceIndex();
|
||||
index.addSource(vo);
|
||||
index.addSource(producer);
|
||||
|
||||
Set<String> patterns = allKafkaPatterns();
|
||||
WritePoint wp = findByPattern(
|
||||
new MqWritePointDetector(index, patterns).detect("PatrolNotifyProducer.java", producer),
|
||||
"MQ-K03");
|
||||
assertEquals("patrol-notify-topic", wp.getResolvedKeyPattern());
|
||||
assertTrue(wp.getResolvedValueType().endsWith("PatrolNotifyVo"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void detectsKafkaJsonStringAsMqK04() {
|
||||
String vo = TestSupport.fixture("fixtures/mq/kafka-record/PatrolNotifyVo.txt");
|
||||
String producer = TestSupport.fixture("fixtures/mq/kafka-record/PatrolNotifyProducer.txt");
|
||||
SourceIndex index = new SourceIndex();
|
||||
index.addSource(vo);
|
||||
index.addSource(producer);
|
||||
|
||||
WritePoint wp = findByPattern(
|
||||
new MqWritePointDetector(index, allKafkaPatterns())
|
||||
.detect("PatrolNotifyProducer.java", producer),
|
||||
"MQ-K04");
|
||||
assertTrue(wp.getResolvedValueType().endsWith("PatrolNotifyVo"));
|
||||
}
|
||||
|
||||
private static Set<String> allRocketPatterns() {
|
||||
Set<String> patterns = new HashSet<>();
|
||||
patterns.add("MQ01");
|
||||
patterns.add("MQ02");
|
||||
patterns.add("MQ03");
|
||||
patterns.add("MQ04");
|
||||
patterns.add("MQ05");
|
||||
return patterns;
|
||||
}
|
||||
|
||||
private static Set<String> allKafkaPatterns() {
|
||||
Set<String> patterns = new HashSet<>();
|
||||
patterns.add("MQ-K01");
|
||||
patterns.add("MQ-K02");
|
||||
patterns.add("MQ-K03");
|
||||
patterns.add("MQ-K04");
|
||||
return patterns;
|
||||
}
|
||||
|
||||
private static WritePoint findByPattern(List<WritePoint> wps, String pattern) {
|
||||
return wps.stream()
|
||||
.filter(w -> pattern.equals(w.getPattern()))
|
||||
.findFirst()
|
||||
.orElseThrow(() -> new AssertionError("缺少模式 " + pattern + ",实际: " + wps));
|
||||
}
|
||||
}
|
||||
@@ -40,6 +40,41 @@ class SchemaDifferTest {
|
||||
assertEquals(2, moved, "dbName 与 linkList[].id 均应迁移");
|
||||
}
|
||||
|
||||
@Test
|
||||
void detectsWrapperRemovalAndUpwardMoves() {
|
||||
TypeSchema oldS = new TypeSchema("CacheEnvelope");
|
||||
oldS.add(new FieldSchema("vo", JsonType.OBJECT, "TenantVO"));
|
||||
oldS.add(new FieldSchema("vo.dbName", JsonType.STRING, "String"));
|
||||
oldS.add(new FieldSchema("vo.linkList", JsonType.ARRAY, "List"));
|
||||
oldS.add(new FieldSchema("vo.linkList[]", JsonType.OBJECT, "TenantLinkModel"));
|
||||
oldS.add(new FieldSchema("vo.linkList[].id", JsonType.STRING, "String"));
|
||||
|
||||
TypeSchema newS = new TypeSchema("TenantVO");
|
||||
newS.add(new FieldSchema("dbName", JsonType.STRING, "String"));
|
||||
newS.add(new FieldSchema("linkList", JsonType.ARRAY, "List"));
|
||||
newS.add(new FieldSchema("linkList[]", JsonType.OBJECT, "TenantLinkModel"));
|
||||
newS.add(new FieldSchema("linkList[].id", JsonType.STRING, "String"));
|
||||
|
||||
List<SchemaChange> changes = new SchemaDiffer().diff(oldS, newS);
|
||||
List<ChangeType> types = changes.stream().map(SchemaChange::getChangeType).collect(Collectors.toList());
|
||||
|
||||
assertTrue(types.contains(ChangeType.WRAPPER_REMOVED), "应检测到包装层删除");
|
||||
assertTrue(types.contains(ChangeType.FIELD_PATH_MOVED), "应检测到字段上提迁移");
|
||||
assertTrue(types.stream().noneMatch(t -> t == ChangeType.FIELD_ADDED),
|
||||
"上提字段不应再被当成新增");
|
||||
assertTrue(types.stream().noneMatch(t -> t == ChangeType.FIELD_REMOVED),
|
||||
"上提字段不应再被当成删除");
|
||||
|
||||
SchemaChange wrapper = changes.stream()
|
||||
.filter(c -> c.getChangeType() == ChangeType.WRAPPER_REMOVED)
|
||||
.findFirst()
|
||||
.orElseThrow(IllegalStateException::new);
|
||||
assertEquals("vo", wrapper.getFieldPath());
|
||||
|
||||
long moved = changes.stream().filter(c -> c.getChangeType() == ChangeType.FIELD_PATH_MOVED).count();
|
||||
assertEquals(2, moved, "vo.dbName / vo.linkList[].id 均应上提");
|
||||
}
|
||||
|
||||
@Test
|
||||
void detectsTypeChange() {
|
||||
TypeSchema oldS = new TypeSchema("A");
|
||||
@@ -51,6 +86,57 @@ class SchemaDifferTest {
|
||||
assertEquals(1, changes.size());
|
||||
assertEquals(ChangeType.TYPE_CHANGED, changes.get(0).getChangeType());
|
||||
assertEquals(Severity.P0, changes.get(0).getSeverity());
|
||||
assertTrue(changes.get(0).getMessage().contains("String"));
|
||||
assertTrue(changes.get(0).getMessage().contains("Integer"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void detectsBigDecimalToStringEvenWhenBothWereStringBucketHistorically() {
|
||||
// 即便 JsonType 同为 STRING,javaType 变化也必须检出(用户真实漏报场景)
|
||||
TypeSchema oldS = new TypeSchema("A");
|
||||
oldS.add(new FieldSchema("amount", JsonType.STRING, "BigDecimal"));
|
||||
TypeSchema newS = new TypeSchema("A");
|
||||
newS.add(new FieldSchema("amount", JsonType.STRING, "String"));
|
||||
|
||||
List<SchemaChange> changes = new SchemaDiffer().diff(oldS, newS);
|
||||
assertEquals(1, changes.size());
|
||||
assertEquals(ChangeType.TYPE_CHANGED, changes.get(0).getChangeType());
|
||||
assertEquals("amount", changes.get(0).getFieldPath());
|
||||
assertTrue(changes.get(0).getOldValue().contains("BigDecimal"));
|
||||
assertTrue(changes.get(0).getNewValue().contains("String"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void detectsBigDecimalToStringViaJsonTypeAndJavaType() {
|
||||
TypeSchema oldS = new TypeSchema("A");
|
||||
oldS.add(new FieldSchema("amount", JsonType.NUMBER, "BigDecimal"));
|
||||
TypeSchema newS = new TypeSchema("A");
|
||||
newS.add(new FieldSchema("amount", JsonType.STRING, "String"));
|
||||
|
||||
List<SchemaChange> changes = new SchemaDiffer().diff(oldS, newS);
|
||||
assertEquals(1, changes.size());
|
||||
assertEquals(ChangeType.TYPE_CHANGED, changes.get(0).getChangeType());
|
||||
}
|
||||
|
||||
@Test
|
||||
void detectsIntegerToLongWithinNumberBucket() {
|
||||
TypeSchema oldS = new TypeSchema("A");
|
||||
oldS.add(new FieldSchema("id", JsonType.NUMBER, "Integer"));
|
||||
TypeSchema newS = new TypeSchema("A");
|
||||
newS.add(new FieldSchema("id", JsonType.NUMBER, "Long"));
|
||||
|
||||
List<SchemaChange> changes = new SchemaDiffer().diff(oldS, newS);
|
||||
assertEquals(1, changes.size());
|
||||
assertEquals(ChangeType.TYPE_CHANGED, changes.get(0).getChangeType());
|
||||
}
|
||||
|
||||
@Test
|
||||
void noChangeWhenPrimitiveAndWrapperEquivalent() {
|
||||
TypeSchema a = new TypeSchema("A");
|
||||
a.add(new FieldSchema("n", JsonType.NUMBER, "int"));
|
||||
TypeSchema b = new TypeSchema("A");
|
||||
b.add(new FieldSchema("n", JsonType.NUMBER, "Integer"));
|
||||
assertTrue(new SchemaDiffer().diff(a, b).isEmpty());
|
||||
}
|
||||
|
||||
@Test
|
||||
|
||||
@@ -65,4 +65,91 @@ class RedisKeyResolverTest {
|
||||
buildCall, clazz, index.get("jnpf.util.TenantDbContentCacheHelper"));
|
||||
assertEquals("tenant:db:content:*", pattern);
|
||||
}
|
||||
|
||||
@Test
|
||||
void resolvesLocalVariableAssignedFromStringFormat() {
|
||||
String source = ""
|
||||
+ "package demo;\n"
|
||||
+ "public class CombinationRepo {\n"
|
||||
+ " private static final String COMBINATION_QUERY_KEY = "
|
||||
+ "\"data_analysis:combination_query_key:%s:%s\";\n"
|
||||
+ " public void write(String tenantId, String hashKey) {\n"
|
||||
+ " String key = String.format(COMBINATION_QUERY_KEY, tenantId, hashKey);\n"
|
||||
+ " redisSet(key);\n"
|
||||
+ " }\n"
|
||||
+ " void redisSet(String k) {}\n"
|
||||
+ "}\n";
|
||||
|
||||
SourceIndex index = new SourceIndex();
|
||||
index.addSource(source);
|
||||
CompilationUnit cu = StaticJavaParser.parse(source);
|
||||
ClassOrInterfaceDeclaration clazz = cu.getType(0).asClassOrInterfaceDeclaration();
|
||||
// redisSet(key) 的实参 key —— 局部变量使用点
|
||||
MethodCallExpr redisSet = cu.findAll(MethodCallExpr.class).stream()
|
||||
.filter(m -> "redisSet".equals(m.getNameAsString()))
|
||||
.findFirst()
|
||||
.orElseThrow(IllegalStateException::new);
|
||||
Expression keyArg = redisSet.getArgument(0);
|
||||
|
||||
String pattern = new RedisKeyResolver(index).resolve(
|
||||
keyArg, clazz, index.get("demo.CombinationRepo"));
|
||||
assertEquals("data_analysis:combination_query_key:*:*", pattern);
|
||||
}
|
||||
|
||||
@Test
|
||||
void localReassignmentUsesNearestBindingBeforeUse() {
|
||||
String source = ""
|
||||
+ "package demo;\n"
|
||||
+ "public class ReassignRepo {\n"
|
||||
+ " private static final String A = \"prefix:a:%s\";\n"
|
||||
+ " private static final String B = \"prefix:b:%s\";\n"
|
||||
+ " public void write(String id) {\n"
|
||||
+ " String key = String.format(A, id);\n"
|
||||
+ " key = String.format(B, id);\n"
|
||||
+ " redisSet(key);\n"
|
||||
+ " }\n"
|
||||
+ " void redisSet(String k) {}\n"
|
||||
+ "}\n";
|
||||
|
||||
SourceIndex index = new SourceIndex();
|
||||
index.addSource(source);
|
||||
CompilationUnit cu = StaticJavaParser.parse(source);
|
||||
ClassOrInterfaceDeclaration clazz = cu.getType(0).asClassOrInterfaceDeclaration();
|
||||
MethodCallExpr redisSet = cu.findAll(MethodCallExpr.class).stream()
|
||||
.filter(m -> "redisSet".equals(m.getNameAsString()))
|
||||
.findFirst()
|
||||
.orElseThrow(IllegalStateException::new);
|
||||
|
||||
String pattern = new RedisKeyResolver(index).resolve(
|
||||
redisSet.getArgument(0), clazz, index.get("demo.ReassignRepo"));
|
||||
assertEquals("prefix:b:*", pattern);
|
||||
}
|
||||
|
||||
@Test
|
||||
void staticConstantStillPreferredOverLocalNameCollision() {
|
||||
// 局部变量名与静态常量同名时:仍优先静态常量(保持旧语义)
|
||||
String source = ""
|
||||
+ "package demo;\n"
|
||||
+ "public class Collision {\n"
|
||||
+ " private static final String KEY = \"const:prefix:%s\";\n"
|
||||
+ " public void write(String id) {\n"
|
||||
+ " String KEY = \"local:\" + id;\n"
|
||||
+ " redisSet(KEY);\n"
|
||||
+ " }\n"
|
||||
+ " void redisSet(String k) {}\n"
|
||||
+ "}\n";
|
||||
|
||||
SourceIndex index = new SourceIndex();
|
||||
index.addSource(source);
|
||||
CompilationUnit cu = StaticJavaParser.parse(source);
|
||||
ClassOrInterfaceDeclaration clazz = cu.getType(0).asClassOrInterfaceDeclaration();
|
||||
MethodCallExpr redisSet = cu.findAll(MethodCallExpr.class).stream()
|
||||
.filter(m -> "redisSet".equals(m.getNameAsString()))
|
||||
.findFirst()
|
||||
.orElseThrow(IllegalStateException::new);
|
||||
|
||||
String pattern = new RedisKeyResolver(index).resolve(
|
||||
redisSet.getArgument(0), clazz, index.get("demo.Collision"));
|
||||
assertEquals("const:prefix:%s", pattern);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,6 +5,7 @@ import com.codechecker.cache.diff.SchemaChange;
|
||||
import com.codechecker.cache.diff.Severity;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.Collections;
|
||||
import java.util.LinkedHashSet;
|
||||
import java.util.List;
|
||||
@@ -60,8 +61,8 @@ class ReportBuilderTest {
|
||||
|
||||
assertTrue(md.contains("- Key --> `saas:period-config:migration:current`"));
|
||||
assertFalse(md.contains("(key 无法解析)"));
|
||||
assertTrue(md.contains("> **位置**: `SaasPeriodConfigMigrationRedisSupport#putCurrent:41`"));
|
||||
assertTrue(md.contains("> **类型**: `MigrationCurrentVo`"));
|
||||
assertTrue(md.contains("> **位置**: SaasPeriodConfigMigrationRedisSupport#putCurrent:41"));
|
||||
assertTrue(md.contains("> **类型**: MigrationCurrentVo"));
|
||||
|
||||
int oldSection = md.indexOf("> **value值由:**");
|
||||
int newSection = md.indexOf("> **变更为:**");
|
||||
@@ -111,6 +112,205 @@ class ReportBuilderTest {
|
||||
assertFalse(newMd.startsWith("`"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void typeChangedHighlightedGrayInBothSkeletons() {
|
||||
SchemaChange typeChanged = new SchemaChange(ChangeType.TYPE_CHANGED);
|
||||
typeChanged.setFieldPath("amount");
|
||||
typeChanged.setOldValue("BigDecimal/NUMBER");
|
||||
typeChanged.setNewValue("String/STRING");
|
||||
typeChanged.setMessage("字段 amount 类型由 BigDecimal/NUMBER 变为 String/STRING");
|
||||
|
||||
String oldJson = "{\"amount\":0,\"name\":\"\"}";
|
||||
String newJson = "{\"amount\":\"\",\"name\":\"\"}";
|
||||
List<SchemaChange> details = Collections.singletonList(typeChanged);
|
||||
|
||||
Set<String> oldPaths = SkeletonAnnotator.pathsForOldSkeleton(details);
|
||||
Set<String> newGreen = SkeletonAnnotator.pathsForNewSkeleton(details);
|
||||
Set<String> typeGray = SkeletonAnnotator.pathsForTypeChanged(details);
|
||||
|
||||
assertTrue(oldPaths.isEmpty());
|
||||
assertTrue(newGreen.isEmpty());
|
||||
assertEquals(Collections.singleton("amount"), typeGray);
|
||||
|
||||
String oldMd = SkeletonAnnotator.annotateOldForWecom(oldJson, oldPaths, typeGray);
|
||||
String newMd = SkeletonAnnotator.annotateNewForWecom(newJson, newGreen, typeGray);
|
||||
|
||||
assertTrue(oldMd.contains("<font color=\"comment\">\"amount\":0</font>"));
|
||||
assertFalse(oldMd.contains("<font color=\"warning\">\"amount\""));
|
||||
assertFalse(oldMd.contains("<font color=\"comment\">\"name\":\"\"</font>"));
|
||||
assertTrue(newMd.contains("<font color=\"comment\">\"amount\":\"\"</font>"));
|
||||
assertFalse(newMd.contains("<font color=\"info\">\"amount\""));
|
||||
assertFalse(newMd.contains("<font color=\"warning\">\"amount\""));
|
||||
assertFalse(newMd.contains("<font color=\"comment\">\"name\":\"\"</font>"));
|
||||
|
||||
CheckReport report = baseReport();
|
||||
KeyStructureChange key = new KeyStructureChange();
|
||||
key.setKeyPattern("cache:amount");
|
||||
key.setWriteLocation("Demo#put:1");
|
||||
key.setValueType("AmountVo");
|
||||
key.setOldSkeletonJson(oldJson);
|
||||
key.setNewSkeletonJson(newJson);
|
||||
key.getFieldDetails().add(typeChanged);
|
||||
report.getKeyChanges().add(key);
|
||||
|
||||
String md = new ReportBuilder("[序列化结构变更]").toMarkdown(report);
|
||||
int oldSection = md.indexOf("> **value值由:**");
|
||||
int newSection = md.indexOf("> **变更为:**");
|
||||
assertTrue(oldSection >= 0);
|
||||
assertTrue(newSection > oldSection);
|
||||
String oldPart = md.substring(oldSection, newSection);
|
||||
String newPart = md.substring(newSection);
|
||||
assertTrue(oldPart.contains("<font color=\"comment\">\"amount\":0</font>"));
|
||||
assertTrue(newPart.contains("<font color=\"comment\">\"amount\":\"\"</font>"));
|
||||
assertFalse(newPart.contains("<font color=\"info\">\"amount\""));
|
||||
assertFalse(oldPart.contains("<font color=\"warning\">\"amount\""));
|
||||
assertTrue(md.contains("> **类型变更**: amount <font color=\"warning\">BigDecimal → String</font>"),
|
||||
"应包含类型变更摘要行,实际 markdown:\n" + md);
|
||||
}
|
||||
|
||||
@Test
|
||||
void typeChangeSummaryDedupesSameFieldFromMultipleWrites() {
|
||||
CheckReport report = baseReport();
|
||||
KeyStructureChange key = new KeyStructureChange();
|
||||
key.setKeyPattern("data-analysis-task-topic");
|
||||
key.setChannel("ROCKETMQ");
|
||||
key.setWriteLocation("FtbDataAnalysisTaskServiceImpl#createTasks:48");
|
||||
key.setValueType("DataAnalysisBO");
|
||||
key.setOldSkeletonJson("{\"test1\":\"\"}");
|
||||
key.setNewSkeletonJson("{\"test1\":0}");
|
||||
|
||||
SchemaChange a = new SchemaChange(ChangeType.TYPE_CHANGED);
|
||||
a.setFieldPath("test1");
|
||||
a.setOldValue("String/STRING");
|
||||
a.setNewValue("Integer/NUMBER");
|
||||
a.setWriteLocation("FtbDataAnalysisTaskServiceImpl#createTasks:48");
|
||||
SchemaChange b = new SchemaChange(ChangeType.TYPE_CHANGED);
|
||||
b.setFieldPath("test1");
|
||||
b.setOldValue("String/STRING");
|
||||
b.setNewValue("Integer/NUMBER");
|
||||
b.setWriteLocation("FtbDataAnalysisTaskServiceImpl#createTasks:52");
|
||||
key.getFieldDetails().add(a);
|
||||
key.getFieldDetails().add(b);
|
||||
report.getKeyChanges().add(key);
|
||||
|
||||
String md = new ReportBuilder("[序列化结构变更]").toMarkdown(report);
|
||||
assertTrue(md.contains("> **类型变更**: test1 <font color=\"warning\">String → Integer</font>"), md);
|
||||
assertFalse(md.contains("String → Integer</font>;test1"),
|
||||
"同字段类型变更不应重复,实际:\n" + md);
|
||||
}
|
||||
|
||||
@Test
|
||||
void wrapperAddedHighlightedBeforeNestedMovedFields() {
|
||||
SchemaChange wrapper = new SchemaChange(ChangeType.WRAPPER_ADDED);
|
||||
wrapper.setFieldPath("vo");
|
||||
wrapper.setMessage("新增包装层 vo");
|
||||
|
||||
SchemaChange movedDb = new SchemaChange(ChangeType.FIELD_PATH_MOVED);
|
||||
movedDb.setFieldPath("vo.dbName");
|
||||
movedDb.setOldValue("dbName");
|
||||
movedDb.setNewValue("vo.dbName");
|
||||
|
||||
SchemaChange movedLink = new SchemaChange(ChangeType.FIELD_PATH_MOVED);
|
||||
movedLink.setFieldPath("vo.linkList[].id");
|
||||
movedLink.setOldValue("linkList[].id");
|
||||
movedLink.setNewValue("vo.linkList[].id");
|
||||
|
||||
List<SchemaChange> details = new ArrayList<>();
|
||||
details.add(wrapper);
|
||||
details.add(movedDb);
|
||||
details.add(movedLink);
|
||||
|
||||
String oldJson = "{\"dbName\":\"\",\"linkList\":[{\"id\":\"\"}]}";
|
||||
String newJson = "{\"vo\":{\"dbName\":\"\",\"linkList\":[{\"id\":\"\"}]}}";
|
||||
|
||||
Set<String> wrappers = SkeletonAnnotator.pathsForWrapperAdded(details);
|
||||
Set<String> newGreen = SkeletonAnnotator.pathsForNewSkeleton(details);
|
||||
assertEquals(Collections.singleton("vo"), wrappers);
|
||||
assertTrue(newGreen.contains("vo.dbName"));
|
||||
assertFalse(newGreen.contains("vo"), "包装层不应再混入普通新增路径");
|
||||
|
||||
// 先标包装层:整段 vo 对象应绿;子路径随后因已在 font 内可跳过
|
||||
String newMd = SkeletonAnnotator.annotateNewForWecom(newJson, newGreen, null, wrappers);
|
||||
assertTrue(newMd.contains("<font color=\"info\">\"vo\":{"), newMd);
|
||||
assertTrue(newMd.contains("</font>"), newMd);
|
||||
|
||||
CheckReport report = baseReport();
|
||||
KeyStructureChange key = new KeyStructureChange();
|
||||
key.setKeyPattern("tenant:db:content:*");
|
||||
key.setWriteLocation("TenantDbContentCacheHelper#cacheNotFound:62");
|
||||
key.setValueType("TenantVO");
|
||||
key.setOldSkeletonJson(oldJson);
|
||||
key.setNewSkeletonJson(newJson);
|
||||
key.getFieldDetails().addAll(details);
|
||||
report.getKeyChanges().add(key);
|
||||
|
||||
String md = new ReportBuilder("[序列化结构变更]").toMarkdown(report);
|
||||
int newSection = md.indexOf("> **变更为:**");
|
||||
assertTrue(newSection >= 0);
|
||||
String newPart = md.substring(newSection);
|
||||
assertTrue(newPart.contains("<font color=\"info\">\"vo\":{"),
|
||||
"包装层 vo 应整段绿色,实际:\n" + newPart);
|
||||
}
|
||||
|
||||
@Test
|
||||
void wrapperRemovedHighlightedBeforeNestedMovedFieldsOnOldSkeleton() {
|
||||
SchemaChange wrapper = new SchemaChange(ChangeType.WRAPPER_REMOVED);
|
||||
wrapper.setFieldPath("vo");
|
||||
wrapper.setOldValue("vo");
|
||||
wrapper.setMessage("删除包装层 vo");
|
||||
|
||||
SchemaChange movedDb = new SchemaChange(ChangeType.FIELD_PATH_MOVED);
|
||||
movedDb.setFieldPath("dbName");
|
||||
movedDb.setOldValue("vo.dbName");
|
||||
movedDb.setNewValue("dbName");
|
||||
|
||||
SchemaChange movedLink = new SchemaChange(ChangeType.FIELD_PATH_MOVED);
|
||||
movedLink.setFieldPath("linkList[].id");
|
||||
movedLink.setOldValue("vo.linkList[].id");
|
||||
movedLink.setNewValue("linkList[].id");
|
||||
|
||||
List<SchemaChange> details = new ArrayList<>();
|
||||
details.add(wrapper);
|
||||
details.add(movedDb);
|
||||
details.add(movedLink);
|
||||
|
||||
String oldJson = "{\"vo\":{\"dbName\":\"\",\"linkList\":[{\"id\":\"\"}]}}";
|
||||
String newJson = "{\"dbName\":\"\",\"linkList\":[{\"id\":\"\"}]}";
|
||||
|
||||
Set<String> wrappersRemoved = SkeletonAnnotator.pathsForWrapperRemoved(details);
|
||||
Set<String> oldOrange = SkeletonAnnotator.pathsForOldSkeleton(details);
|
||||
Set<String> newGreen = SkeletonAnnotator.pathsForNewSkeleton(details);
|
||||
|
||||
assertEquals(Collections.singleton("vo"), wrappersRemoved);
|
||||
assertTrue(oldOrange.contains("vo.dbName"));
|
||||
assertTrue(newGreen.contains("dbName"));
|
||||
assertFalse(newGreen.contains("vo"));
|
||||
|
||||
String oldMd = SkeletonAnnotator.annotateOldForWecom(oldJson, oldOrange, null, wrappersRemoved);
|
||||
assertTrue(oldMd.contains("<font color=\"warning\">\"vo\":{"), oldMd);
|
||||
|
||||
String newMd = SkeletonAnnotator.annotateNewForWecom(newJson, newGreen, null, null);
|
||||
assertTrue(newMd.contains("<font color=\"info\">\"dbName\":\"\"</font>"), newMd);
|
||||
assertFalse(newMd.contains("<font color=\"info\">\"vo\""), newMd);
|
||||
|
||||
CheckReport report = baseReport();
|
||||
KeyStructureChange key = new KeyStructureChange();
|
||||
key.setKeyPattern("tenant:db:content:*");
|
||||
key.setWriteLocation("TenantDbContentCacheHelper#cacheNotFound:62");
|
||||
key.setValueType("TenantVO");
|
||||
key.setOldSkeletonJson(oldJson);
|
||||
key.setNewSkeletonJson(newJson);
|
||||
key.getFieldDetails().addAll(details);
|
||||
report.getKeyChanges().add(key);
|
||||
|
||||
String md = new ReportBuilder("[序列化结构变更]").toMarkdown(report);
|
||||
int oldSection = md.indexOf("> **value值由:**");
|
||||
int newSection = md.indexOf("> **变更为:**");
|
||||
assertTrue(oldSection >= 0 && newSection > oldSection);
|
||||
assertTrue(md.substring(oldSection, newSection).contains("<font color=\"warning\">\"vo\":{"),
|
||||
"旧骨架应整段橙标 vo,实际:\n" + md);
|
||||
}
|
||||
|
||||
@Test
|
||||
void consoleContainsFieldDetailsAndWecomMarkdown() {
|
||||
CheckReport report = new CheckReport();
|
||||
@@ -138,8 +338,8 @@ class ReportBuilderTest {
|
||||
assertTrue(console.contains("**删除字段**: x"));
|
||||
assertTrue(console.contains("<font color=\"warning\">\"x\":\"\"</font>"));
|
||||
assertFalse(console.contains("<font color=\"info\">\"x\":\"\"</font>"));
|
||||
assertTrue(console.contains("> **位置**: `"));
|
||||
assertTrue(console.contains("> **类型**: `"));
|
||||
assertTrue(console.contains("> **位置**: "));
|
||||
assertTrue(console.contains("> **类型**: "));
|
||||
assertTrue(console.contains("> **value值由:**"));
|
||||
assertTrue(console.contains("> **变更为:**"));
|
||||
assertFalse(console.contains("`{\"x\":\"\"}`"));
|
||||
@@ -163,8 +363,8 @@ class ReportBuilderTest {
|
||||
|
||||
String md = new ReportBuilder("[序列化结构变更]").toMarkdown(report);
|
||||
assertTrue(md.contains("- Key --> `req.getKey()` <font color=\"comment\">(key 无法解析)</font>"));
|
||||
assertTrue(md.contains("> **位置**: `ClockInXxxService#export:128`"));
|
||||
assertTrue(md.contains("> **类型**: `List<ClockInExportVo>`"));
|
||||
assertTrue(md.contains("> **位置**: ClockInXxxService#export:128"));
|
||||
assertTrue(md.contains("> **类型**: List<ClockInExportVo>"));
|
||||
assertFalse(md.contains("`unknown-key`"));
|
||||
}
|
||||
|
||||
@@ -216,6 +416,24 @@ class ReportBuilderTest {
|
||||
assertTrue(console.contains("超长已按 key 拆为 2 条"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void wildcardAsteriskInKeyEscapedForWeCom() {
|
||||
assertEquals("data_analysis:combination_query_key:*:*",
|
||||
ReportBuilder.escapeWeComCode("data_analysis:combination_query_key:*:*"));
|
||||
|
||||
CheckReport report = baseReport();
|
||||
KeyStructureChange key = simpleKey(
|
||||
"data_analysis:combination_query_key:*:*",
|
||||
"{\"a\":0}", "{\"a\":0,\"b\":0}", "b");
|
||||
report.getKeyChanges().add(key);
|
||||
|
||||
String md = new ReportBuilder("[序列化结构变更]").toMarkdown(report);
|
||||
assertTrue(md.contains("- Key --> `data_analysis:combination_query_key:*:*`"),
|
||||
"通配符 * 应转成全角*,实际:\n" + md);
|
||||
assertFalse(md.contains("combination_query_key:*:*"),
|
||||
"不应再输出未转义的 *:*, 实际:\n" + md);
|
||||
}
|
||||
|
||||
private static CheckReport baseReport() {
|
||||
CheckReport report = new CheckReport();
|
||||
report.setRepository("jnpf-java-cloud");
|
||||
|
||||
@@ -81,4 +81,21 @@ class JavaSchemaExtractorTest {
|
||||
assertTrue(AnnotationSupport.ignoredProperties(type).contains("a"));
|
||||
assertTrue(AnnotationSupport.ignoredProperties(type).contains("b"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void bigDecimalMappedAsNumberNotString() {
|
||||
String source = ""
|
||||
+ "package demo;\n"
|
||||
+ "import java.math.BigDecimal;\n"
|
||||
+ "public class MoneyVo {\n"
|
||||
+ " private BigDecimal amount;\n"
|
||||
+ " private String currency;\n"
|
||||
+ "}\n";
|
||||
SourceIndex index = new SourceIndex();
|
||||
index.addSource(source);
|
||||
TypeSchema schema = new JavaSchemaExtractor(index, 8).extract("demo.MoneyVo");
|
||||
assertEquals(JsonType.NUMBER, schema.getFields().get("amount").getJsonType());
|
||||
assertEquals("BigDecimal", schema.getFields().get("amount").getJavaType());
|
||||
assertEquals(JsonType.STRING, schema.getFields().get("currency").getJsonType());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -80,4 +80,31 @@ class SkeletonJsonRendererTest {
|
||||
"截断后仍应保留改动相关字段: " + truncated);
|
||||
assertEquals(true, truncated.length() >= 10);
|
||||
}
|
||||
|
||||
@Test
|
||||
void numberPlaceholdersDistinguishIntegerFloatAndBigDecimal() {
|
||||
TypeSchema schema = new TypeSchema("MoneyVo");
|
||||
schema.add(new FieldSchema("count", JsonType.NUMBER, "Integer"));
|
||||
schema.add(new FieldSchema("rate", JsonType.NUMBER, "Double"));
|
||||
schema.add(new FieldSchema("amount", JsonType.NUMBER, "BigDecimal"));
|
||||
schema.add(new FieldSchema("score", JsonType.NUMBER, "Float"));
|
||||
|
||||
String json = new SkeletonJsonRenderer().render(schema);
|
||||
assertTrue(json.contains("\"count\":0"), json);
|
||||
assertTrue(json.contains("\"rate\":0.0"), json);
|
||||
assertTrue(json.contains("\"amount\":0.00"), json);
|
||||
assertTrue(json.contains("\"score\":0.0"), json);
|
||||
}
|
||||
|
||||
@Test
|
||||
void integerToBigDecimalSkeletonsLookDifferent() {
|
||||
TypeSchema oldS = new TypeSchema("A");
|
||||
oldS.add(new FieldSchema("amount", JsonType.NUMBER, "Integer"));
|
||||
TypeSchema newS = new TypeSchema("A");
|
||||
newS.add(new FieldSchema("amount", JsonType.NUMBER, "BigDecimal"));
|
||||
|
||||
SkeletonJsonRenderer renderer = new SkeletonJsonRenderer();
|
||||
assertEquals("{\"amount\":0}", renderer.render(oldS));
|
||||
assertEquals("{\"amount\":0.00}", renderer.render(newS));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
package demo.kafka;
|
||||
|
||||
public class CheckItemDetailVo {
|
||||
private String itemId;
|
||||
private String itemName;
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
package demo.kafka;
|
||||
|
||||
public class CheckItemDetailVo {
|
||||
private String itemId;
|
||||
private String itemName;
|
||||
private Integer score;
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
package demo.kafka;
|
||||
|
||||
import java.util.List;
|
||||
import org.springframework.kafka.core.KafkaTemplate;
|
||||
|
||||
public class PatrolService {
|
||||
private KafkaTemplate kafkaTemplate;
|
||||
private static final String TOPIC = "patrol-store-food-safe:%s";
|
||||
|
||||
public void sendFoodSafeData(String tenantId, List<CheckItemDetailVo> thousandsData) {
|
||||
String topic = String.format(TOPIC, tenantId);
|
||||
kafkaTemplate.send(topic, thousandsData);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
package demo.kafka;
|
||||
|
||||
import org.springframework.kafka.annotation.KafkaListener;
|
||||
|
||||
public class PatrolNotifyListener {
|
||||
@KafkaListener(topics = "patrol-notify-topic", groupId = "patrol-group")
|
||||
public void handle(String message) {
|
||||
PatrolNotifyVo vo = JSONObject.parseObject(message, PatrolNotifyVo.class);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
package demo.kafka;
|
||||
|
||||
import org.apache.kafka.clients.producer.ProducerRecord;
|
||||
import org.springframework.kafka.core.KafkaTemplate;
|
||||
|
||||
public class PatrolNotifyProducer {
|
||||
private KafkaTemplate kafkaTemplate;
|
||||
private static final String TOPIC = "patrol-notify-topic";
|
||||
|
||||
public void sendRecord(PatrolNotifyVo vo) {
|
||||
ProducerRecord<String, PatrolNotifyVo> record = new ProducerRecord<>(TOPIC, vo);
|
||||
kafkaTemplate.send(record);
|
||||
}
|
||||
|
||||
public void sendJson(PatrolNotifyVo vo) {
|
||||
String json = JSON.toJSONString(vo);
|
||||
kafkaTemplate.send(TOPIC, json);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
package demo.kafka;
|
||||
|
||||
public class PatrolNotifyVo {
|
||||
private String storeId;
|
||||
private String status;
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
package demo.mq;
|
||||
|
||||
public class DutyImNotice {
|
||||
private String tenantId;
|
||||
private String id;
|
||||
private String content;
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
package demo.mq;
|
||||
|
||||
import org.apache.rocketmq.spring.annotation.RocketMQMessageListener;
|
||||
import org.apache.rocketmq.spring.core.RocketMQListener;
|
||||
|
||||
@RocketMQMessageListener(topic = DutyImNoticeProducer.MQ_TOPIC, consumerGroup = "duty-im-group")
|
||||
public class DutyImNoticeConsumer implements RocketMQListener<DutyImNotice> {
|
||||
public void onMessage(DutyImNotice message) {
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
package demo.mq;
|
||||
|
||||
import org.apache.rocketmq.spring.core.RocketMQTemplate;
|
||||
import org.springframework.messaging.Message;
|
||||
import org.springframework.messaging.support.MessageBuilder;
|
||||
|
||||
public class DutyImNoticeProducer {
|
||||
private RocketMQTemplate rocketMQTemplate;
|
||||
public static final String MQ_TOPIC = "duty-im-notice-topic";
|
||||
|
||||
public void addMq(DutyImNotice message, long timestamp) {
|
||||
String uniqueKey = message.getTenantId() + "_" + message.getId();
|
||||
Message<DutyImNotice> mqMessage = MessageBuilder.withPayload(message)
|
||||
.setHeader("KEYS", uniqueKey)
|
||||
.setHeader("executeTime", timestamp)
|
||||
.build();
|
||||
rocketMQTemplate.asyncSend(MQ_TOPIC, mqMessage);
|
||||
}
|
||||
|
||||
public void convertSend(DutyImNotice message) {
|
||||
rocketMQTemplate.convertAndSend(MQ_TOPIC, message);
|
||||
}
|
||||
|
||||
public void sendJson(DutyImNotice message) {
|
||||
String body = JSON.toJSONString(message);
|
||||
rocketMQTemplate.syncSend(MQ_TOPIC, body);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
package demo.mq;
|
||||
|
||||
public final class CapitalMqConstants {
|
||||
public static final String TOPIC = "capital-topic";
|
||||
public static final String TAG_WALLET_DEDUCT = "WALLET_DEDUCT";
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
package demo.mq;
|
||||
|
||||
import org.apache.rocketmq.spring.core.RocketMQTemplate;
|
||||
|
||||
public class WalletDeductProducer {
|
||||
private RocketMQTemplate rocketMQTemplate;
|
||||
|
||||
public void send(WalletDeductReq req) {
|
||||
rocketMQTemplate.syncSend(CapitalMqConstants.TOPIC + ":" + CapitalMqConstants.TAG_WALLET_DEDUCT, req);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
package demo.mq;
|
||||
|
||||
public class WalletDeductReq {
|
||||
private String walletId;
|
||||
private Long amount;
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
package demo.mq;
|
||||
|
||||
public class WalletDeductReq {
|
||||
private String walletId;
|
||||
private Long amount;
|
||||
private String remark;
|
||||
}
|
||||
Reference in New Issue
Block a user