feat: 流水线更新,兼容多次commit
All checks were successful
缓存序列化结构检查 / cache-schema-check (push) Has been skipped
All checks were successful
缓存序列化结构检查 / cache-schema-check (push) Has been skipped
This commit is contained in:
@@ -15,14 +15,19 @@ jobs:
|
||||
if: ${{ gitea.ref != 'refs/heads/pre' && gitea.ref != 'refs/heads/dev' && gitea.ref != 'refs/heads/master-2.0' }}
|
||||
runs-on: jdk11
|
||||
steps:
|
||||
# 浅克隆:仅需 HEAD 与 HEAD~1,避免全量拉取
|
||||
# 指定 --branch 确保非默认分支推送时也能正确检出 gitea.sha
|
||||
# 浅克隆当前 tip;对比基准用 push before(覆盖一次 push 的多 commit 累计 diff)
|
||||
# 不必全量历史:只需能 git show before / after 两边的文件内容
|
||||
- name: 检出代码
|
||||
run: |
|
||||
git config --global http.sslVerify false
|
||||
git clone --depth 2 --single-branch --branch "${{ gitea.ref_name }}" \
|
||||
"https://${{ gitea.token }}@git.niujiekeji.com/${{ gitea.repository }}.git" .
|
||||
git checkout -B "${{ gitea.ref_name }}" "${{ gitea.sha }}"
|
||||
REPO_URL="https://${{ gitea.token }}@git.niujiekeji.com/${{ gitea.repository }}.git"
|
||||
BRANCH="${{ gitea.ref_name }}"
|
||||
NEW_SHA="${{ gitea.sha }}"
|
||||
|
||||
git clone --depth 1 --single-branch --branch "${BRANCH}" "${REPO_URL}" .
|
||||
git checkout -B "${BRANCH}" "${NEW_SHA}"
|
||||
|
||||
echo "${NEW_SHA}" > /tmp/cache-schema-new-sha.txt
|
||||
|
||||
- name: 检查配置文件
|
||||
run: |
|
||||
@@ -74,25 +79,81 @@ jobs:
|
||||
echo "Java: $(java -version 2>&1 | head -1)"
|
||||
|
||||
- name: 执行缓存序列化结构检测
|
||||
env:
|
||||
# push 前 tip;新分支首次 push 时为全 0。workflow_dispatch 可能为空,下方会回退。
|
||||
PUSH_BEFORE: ${{ gitea.event.before }}
|
||||
run: |
|
||||
if [ -f /tmp/cache-schema-check.skip ]; then
|
||||
echo "总开关已关闭,跳过检测"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
NEW_SHA=$(cat /tmp/cache-schema-new-sha.txt)
|
||||
OLD_SHA="${PUSH_BEFORE}"
|
||||
|
||||
# 新分支首次 push(before 全 0)→ 跳过
|
||||
if [ -z "$OLD_SHA" ] || echo "$OLD_SHA" | grep -Eq '^0+$'; then
|
||||
# 手动触发:无 before,回退为 HEAD~1(加深 1 层后取父提交)
|
||||
if [ "${{ gitea.event_name }}" = "workflow_dispatch" ]; then
|
||||
git fetch --deepen 1 || true
|
||||
OLD_SHA=$(git rev-parse HEAD~1 2>/dev/null || echo "")
|
||||
if [ -z "$OLD_SHA" ]; then
|
||||
echo "首次提交,跳过检测"
|
||||
echo "手动触发且无父提交,跳过检测"
|
||||
exit 0
|
||||
fi
|
||||
echo "workflow_dispatch:无 push before,回退使用 HEAD~1=${OLD_SHA}"
|
||||
else
|
||||
echo "新分支首次 push(before 为空/全 0),跳过检测"
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$OLD_SHA" = "$NEW_SHA" ]; then
|
||||
echo "before 与 after 相同,无需检测"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
COMMIT_TIME=$(git log -1 --format=%cd --date=format:'%Y-%m-%d %H:%M:%S')
|
||||
# 确保 before 提交对象可读(按 SHA 浅取,无需全量历史)
|
||||
ensure_commit() {
|
||||
local sha="$1"
|
||||
if git cat-file -e "${sha}^{commit}" 2>/dev/null; then
|
||||
return 0
|
||||
fi
|
||||
echo "本地缺少 ${sha},尝试按 SHA 浅取…"
|
||||
if git fetch --depth 1 origin "${sha}"; then
|
||||
git cat-file -e "${sha}^{commit}" 2>/dev/null && return 0
|
||||
fi
|
||||
# 兜底:逐步 deepen(应对部分服务端不允许直接 fetch SHA)
|
||||
for d in 10 30 50 100; do
|
||||
echo "deepen ${d}…"
|
||||
git fetch --deepen "${d}" || true
|
||||
if git cat-file -e "${sha}^{commit}" 2>/dev/null; then
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
if ! ensure_commit "$OLD_SHA"; then
|
||||
echo "错误: 无法获取 push 前 tip ${OLD_SHA},请检查 shallow/权限"
|
||||
exit 2
|
||||
fi
|
||||
if ! ensure_commit "$NEW_SHA"; then
|
||||
echo "错误: 无法解析当前 tip ${NEW_SHA}"
|
||||
exit 2
|
||||
fi
|
||||
|
||||
echo "对比区间: ${OLD_SHA} → ${NEW_SHA}"
|
||||
COMMIT_COUNT=$(git rev-list --count "${OLD_SHA}..${NEW_SHA}" 2>/dev/null || echo "?")
|
||||
echo "本次 push 累计 commit 数(约): ${COMMIT_COUNT}"
|
||||
|
||||
COMMIT_TIME=$(git log -1 --format=%cd --date=format:'%Y-%m-%d %H:%M:%S' "${NEW_SHA}")
|
||||
|
||||
java -jar "/tmp/cache-schema-checker-${CACHE_SCHEMA_CHECKER_VERSION}.jar" \
|
||||
--config .gitea/config/cache-schema-check-config.yaml \
|
||||
--repo-root . \
|
||||
--old-sha "$OLD_SHA" \
|
||||
--new-sha "$(git rev-parse HEAD)" \
|
||||
--new-sha "$NEW_SHA" \
|
||||
--branch "${{ gitea.ref_name }}" \
|
||||
--modifier "${{ gitea.actor }}" \
|
||||
--modify-time "$COMMIT_TIME"
|
||||
|
||||
146
docs/CI集成说明.md
146
docs/CI集成说明.md
@@ -5,21 +5,38 @@
|
||||
## 1. 集成概览
|
||||
|
||||
```text
|
||||
开发者 push 代码
|
||||
开发者 push 代码(可含多个 commit)
|
||||
↓
|
||||
Gitea Actions 触发
|
||||
↓
|
||||
浅克隆业务仓库(depth=2)
|
||||
浅克隆业务仓库 tip(depth=1)+ 按需取 push 前 tip(before)
|
||||
↓
|
||||
从 Nexus 下载 cache-schema-checker.jar
|
||||
↓
|
||||
java -jar 执行(对比 HEAD~1 与 HEAD)
|
||||
java -jar 执行(对比 before → after,累计 diff)
|
||||
↓
|
||||
有结构变更 → 企微通知(按 Key 骨架;删除橙/新增绿)
|
||||
↓
|
||||
mode=block 且含任意结构变更 → exit 1(流水线失败)
|
||||
```
|
||||
|
||||
### 1.1 对比区间(重要)
|
||||
|
||||
| 参数 | 取值 | 含义 |
|
||||
|------|------|------|
|
||||
| `--old-sha` | `gitea.event.before` | 本次 push **前**的远端 tip |
|
||||
| `--new-sha` | `gitea.sha` | 本次 push **后**的 tip |
|
||||
|
||||
一次 push 推了多个 commit 时,只跑 **一次** 检测,覆盖整次 push 的累计代码差,**不会**因为「变更发生在中间 commit、最后一个 commit 没改相关文件」而漏检。
|
||||
|
||||
不需要全量历史:工作树是当前 tip;`before` 通过 `git fetch --depth 1 <sha>`(或 deepen)取到即可。
|
||||
|
||||
边界:
|
||||
|
||||
- `before` 全 `0` / 空 → 新分支首次 push,跳过
|
||||
- `workflow_dispatch` 无 before → 回退 `HEAD~1`
|
||||
- 中间 commit 改坏又被末 commit 改回 → 累计可能无告警(以最终结构为准)
|
||||
|
||||
---
|
||||
|
||||
## 2. 前置条件
|
||||
@@ -45,88 +62,31 @@ jnpf-java-cloud/
|
||||
│ └── cache-schema-check-config.yaml # 检测配置
|
||||
```
|
||||
|
||||
请以本仓库 `.gitea/workflows/cache-schema-check.yaml` 为模板同步到业务仓。
|
||||
|
||||
---
|
||||
|
||||
## 4. 流水线模板
|
||||
## 4. 流水线模板(要点)
|
||||
|
||||
```yaml
|
||||
name: 缓存序列化结构检查
|
||||
run-name: ${{ gitea.actor }}的缓存结构检查
|
||||
完整可运行版本见:`.gitea/workflows/cache-schema-check.yaml`。
|
||||
|
||||
on:
|
||||
push:
|
||||
workflow_dispatch:
|
||||
核心逻辑摘要:
|
||||
|
||||
env:
|
||||
CACHE_SCHEMA_CHECKER_VERSION: "1.0.0"
|
||||
CACHE_SCHEMA_CHECKER_REPO_URL: "http://192.168.3.25:18081/nexus/repository/maven-releases"
|
||||
```bash
|
||||
# 1) 浅拉 tip
|
||||
git clone --depth 1 --single-branch --branch "$BRANCH" "$REPO_URL" .
|
||||
git checkout -B "$BRANCH" "$NEW_SHA" # NEW_SHA = gitea.sha
|
||||
|
||||
jobs:
|
||||
cache-schema-check:
|
||||
if: ${{ gitea.ref != 'refs/heads/pre' && gitea.ref != 'refs/heads/dev' && gitea.ref != 'refs/heads/master-2.0' }}
|
||||
runs-on: jdk11
|
||||
steps:
|
||||
- name: 检出代码
|
||||
run: |
|
||||
git config --global http.sslVerify false
|
||||
git clone --depth 2 --single-branch --branch "${{ gitea.ref_name }}" \
|
||||
"https://${{ gitea.token }}@git.niujiekeji.com/${{ gitea.repository }}.git" .
|
||||
git checkout -B "${{ gitea.ref_name }}" "${{ gitea.sha }}"
|
||||
# 2) OLD_SHA = gitea.event.before(全 0 则跳过;手动触发回退 HEAD~1)
|
||||
# 3) 本地没有 OLD_SHA 时:
|
||||
git fetch --depth 1 origin "$OLD_SHA" # 优先
|
||||
# 或 git fetch --deepen N # 兜底
|
||||
|
||||
- name: 检查配置文件
|
||||
run: |
|
||||
if [ ! -f .gitea/config/cache-schema-check-config.yaml ]; then
|
||||
echo "错误: 缺少 .gitea/config/cache-schema-check-config.yaml"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: 从 Nexus 下载检测工具
|
||||
run: |
|
||||
GROUP_PATH="com/codechecker/cache-schema-checker"
|
||||
JAR_NAME="cache-schema-checker-${CACHE_SCHEMA_CHECKER_VERSION}.jar"
|
||||
JAR_URL="${CACHE_SCHEMA_CHECKER_REPO_URL}/${GROUP_PATH}/${CACHE_SCHEMA_CHECKER_VERSION}/${JAR_NAME}"
|
||||
JAR_PATH="/tmp/${JAR_NAME}"
|
||||
|
||||
echo "下载: ${JAR_URL}"
|
||||
if command -v curl >/dev/null 2>&1; then
|
||||
curl -fsSL -o "${JAR_PATH}" "${JAR_URL}"
|
||||
elif command -v wget >/dev/null 2>&1; then
|
||||
wget -q -O "${JAR_PATH}" "${JAR_URL}"
|
||||
else
|
||||
echo "错误: Runner 缺少 curl 或 wget"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ ! -s "${JAR_PATH}" ]; then
|
||||
echo "错误: 下载失败或文件为空"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
ls -lh "${JAR_PATH}"
|
||||
|
||||
- name: 验证 JDK
|
||||
run: java -version
|
||||
|
||||
- name: 执行 缓存结构检测
|
||||
env:
|
||||
WECOM_ROBOT_WEBHOOK: ${{ secrets.WECOM_ROBOT_WEBHOOK }}
|
||||
run: |
|
||||
OLD_SHA=$(git rev-parse HEAD~1 2>/dev/null || echo "")
|
||||
if [ -z "$OLD_SHA" ]; then
|
||||
echo "首次提交,跳过检测"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
COMMIT_TIME=$(git log -1 --format=%cd --date=format:'%Y-%m-%d %H:%M:%S')
|
||||
|
||||
java -jar "/tmp/cache-schema-checker-${CACHE_SCHEMA_CHECKER_VERSION}.jar" \
|
||||
--config .gitea/config/cache-schema-check-config.yaml \
|
||||
--repo-root . \
|
||||
# 4) 执行
|
||||
java -jar cache-schema-checker-1.0.0.jar \
|
||||
--old-sha "$OLD_SHA" \
|
||||
--new-sha "$(git rev-parse HEAD)" \
|
||||
--branch "${{ gitea.ref_name }}" \
|
||||
--modifier "${{ gitea.actor }}" \
|
||||
--modify-time "$COMMIT_TIME"
|
||||
--new-sha "$NEW_SHA" \
|
||||
...
|
||||
```
|
||||
|
||||
---
|
||||
@@ -148,8 +108,8 @@ jobs:
|
||||
# 在 redisCheck 仓库根目录
|
||||
mvn clean package -DskipTests
|
||||
|
||||
# 发布到 Nexus(需配置 settings.xml,并开启 deploy-nexus profile)
|
||||
mvn clean deploy -Dnexus.deploy.enabled=true -DskipTests
|
||||
# 发布到 Nexus(需配置 settings.xml)
|
||||
mvn clean deploy -DskipTests
|
||||
```
|
||||
|
||||
发布产物:
|
||||
@@ -166,9 +126,9 @@ com/codechecker/cache-schema-checker/1.0.0/
|
||||
|
||||
| 退出码 | 含义 | 流水线表现 |
|
||||
|--------|------|------------|
|
||||
| 0 | 通过(含 notify 模式下的告警) | 绿色 |
|
||||
| 0 | 通过(含 notify 模式下的告警) / 跳过 | 绿色 |
|
||||
| 1 | 阻断(block 模式) | 红色 |
|
||||
| 2 | 执行错误(配置缺失、jar 异常等) | 红色 |
|
||||
| 2 | 执行错误(配置缺失、无法取 before、jar 异常等) | 红色 |
|
||||
|
||||
---
|
||||
|
||||
@@ -176,27 +136,35 @@ com/codechecker/cache-schema-checker/1.0.0/
|
||||
|
||||
| 现象 | 可能原因 | 处理 |
|
||||
|------|----------|------|
|
||||
| 首次提交跳过 | 无 HEAD~1 | 正常行为 |
|
||||
| 新分支首次 push 跳过 | `before` 全 0 | 正常行为 |
|
||||
| 无法获取 before / exit 2 | 浅克隆未取到对象、服务端禁 fetch SHA | 看日志中的 deepen;确认 Gitea 允许按 SHA fetch |
|
||||
| 下载 jar 失败 | Nexus 地址/版本错误 | 检查 env 变量 |
|
||||
| 未收到企微 | Secret 未配 / notify.enabled=false | 检查配置 |
|
||||
| 未收到企微 | Secret 未配 / notify.enabled=false / webhook_url 空 | 检查配置 |
|
||||
| 大量误报 | 锁/计数器未过滤 | 补充 ignore.key_patterns |
|
||||
| 漏报 | 写入模式未覆盖 / 模块过滤过窄 | 确认 W01~W05 已启用;检查 `include_modules` |
|
||||
| 漏报(多 commit) | 业务仓仍用旧版 `HEAD~1` 流水线 | 同步本仓库最新 workflow(`before..after`) |
|
||||
| 漏报(模式/模块) | W0x 未开 / include_modules 过窄 | 确认 W01~W05;检查模块过滤 |
|
||||
| 类型展开不完整 | 类型在依赖 jar 中 | 补充 `manual_mappings.value_type` |
|
||||
|
||||
---
|
||||
|
||||
## 9. 本地调试
|
||||
|
||||
模拟一次「多 commit push」的累计区间:
|
||||
|
||||
```bash
|
||||
# 在 jnpf-java-cloud 根目录
|
||||
# OLD = 推送前 tip,NEW = 当前 tip(可用 origin/branch@{1} 或显式 sha)
|
||||
OLD_SHA=$(git rev-parse origin/$(git branch --show-current)~3) # 示例:假设 ahead 3
|
||||
NEW_SHA=$(git rev-parse HEAD)
|
||||
|
||||
java -jar /path/to/cache-schema-checker-1.0.0.jar \
|
||||
--config .gitea/config/cache-schema-check-config.yaml \
|
||||
--repo-root . \
|
||||
--old-sha HEAD~1 \
|
||||
--new-sha HEAD \
|
||||
--old-sha "$OLD_SHA" \
|
||||
--new-sha "$NEW_SHA" \
|
||||
--branch $(git branch --show-current) \
|
||||
--modifier "$(git log -1 --format=%an)" \
|
||||
--modify-time "$(git log -1 --format=%cd --date=format:'%Y-%m-%d %H:%M:%S')"
|
||||
--modify-time "$(git log -1 --format=%cd --date=format:'%Y-%m-%d %H:%M:%S')" \
|
||||
--dry-run
|
||||
```
|
||||
|
||||
可加 `--dry-run` 仅输出报告不发企微。
|
||||
单 commit 自测仍可用 `--old-sha HEAD~1 --new-sha HEAD`。
|
||||
|
||||
59
docs/实施方案.md
59
docs/实施方案.md
@@ -239,13 +239,18 @@ java -jar cache-schema-checker-1.0.0.jar \
|
||||
|
||||
### 6.2 对比基准(old-sha)获取策略
|
||||
|
||||
与 `demo.yaml` 保持一致,优先级:
|
||||
流水线使用 **push 区间累计对比**(方案:`before` → `after`):
|
||||
|
||||
1. 流水线显式传入 `--old-sha`(通常为 `HEAD~1`)
|
||||
2. 若 `HEAD~1` 不存在(首次提交)→ 跳过检测,`exit 0`
|
||||
3. 浅克隆 `--depth 2` 确保 `HEAD~1` 可用
|
||||
1. `--new-sha` = `gitea.sha`(push 后 tip)
|
||||
2. `--old-sha` = `gitea.event.before`(push 前 tip)
|
||||
3. `before` 为空或全 `0`(新分支首次 push)→ 跳过检测,`exit 0`
|
||||
4. `workflow_dispatch` 无 before 时回退 `HEAD~1`
|
||||
5. 浅克隆当前 tip(`--depth 1`),再按需 `git fetch --depth 1 <before>` / `deepen`,**无需全量历史**
|
||||
|
||||
> 不支持一次 push 多个 commit 时逐个分析;第一版仅对比 `HEAD~1..HEAD`。后续可扩展为 `before..after` 范围分析。
|
||||
> 一次 push 含多个 commit 时,只做 **一次** 检测,对比区间为整次 push 的累计 diff(`before..after`),不会漏掉中间 commit 留下的结构变更。
|
||||
> 不做「每个 commit 各告警一条」;中间引入又被末 commit 改回的净无变更,累计结果可能为「无变更」(符合阻断「最终结构」的目标)。
|
||||
|
||||
详见 `docs/CI集成说明.md`。
|
||||
|
||||
### 6.3 处理步骤
|
||||
|
||||
@@ -465,42 +470,14 @@ notify:
|
||||
详见 `docs/CI集成说明.md`。核心流程:
|
||||
|
||||
```yaml
|
||||
# jnpf-java-cloud/.gitea/workflows/cache-schema-check.yaml
|
||||
name: 缓存序列化结构检查
|
||||
on: [push]
|
||||
|
||||
jobs:
|
||||
cache-schema-check:
|
||||
if: ${{ gitea.ref != 'refs/heads/pre' && gitea.ref != 'refs/heads/dev' && gitea.ref != 'refs/heads/master-2.0' }}
|
||||
runs-on: jdk11
|
||||
steps:
|
||||
- name: 检出代码
|
||||
run: |
|
||||
git clone --depth 2 --single-branch --branch "${{ gitea.ref_name }}" \
|
||||
"https://${{ gitea.token }}@git.niujiekeji.com/${{ gitea.repository }}.git" .
|
||||
git checkout -B "${{ gitea.ref_name }}" "${{ gitea.sha }}"
|
||||
|
||||
- name: 下载检测工具
|
||||
run: |
|
||||
# 从 Nexus 下载 cache-schema-checker jar
|
||||
...
|
||||
|
||||
- name: 执行检测
|
||||
env:
|
||||
WECOM_ROBOT_WEBHOOK: ${{ secrets.WECOM_ROBOT_WEBHOOK }}
|
||||
run: |
|
||||
OLD_SHA=$(git rev-parse HEAD~1 2>/dev/null || echo "")
|
||||
[ -z "$OLD_SHA" ] && exit 0
|
||||
java -jar /tmp/cache-schema-checker-1.0.0.jar \
|
||||
--config .gitea/config/cache-schema-check-config.yaml \
|
||||
--repo-root . \
|
||||
--old-sha "$OLD_SHA" \
|
||||
--new-sha "$(git rev-parse HEAD)" \
|
||||
--branch "${{ gitea.ref_name }}" \
|
||||
--modifier "${{ gitea.actor }}" \
|
||||
--modify-time "$(git log -1 --format=%cd --date=format:'%Y-%m-%d %H:%M:%S')"
|
||||
# jnpf-java-cloud/.gitea/workflows/cache-schema-check.yaml(要点)
|
||||
# 检出:浅克隆 tip(depth 1)
|
||||
# 检测:--old-sha = gitea.event.before,--new-sha = gitea.sha
|
||||
# 按需 fetch before 提交对象,覆盖一次 push 的多 commit 累计 diff
|
||||
```
|
||||
|
||||
完整模板见 `docs/CI集成说明.md` / `.gitea/workflows/cache-schema-check.yaml`。
|
||||
|
||||
---
|
||||
|
||||
## 10. 分阶段交付计划
|
||||
@@ -579,8 +556,8 @@ jobs:
|
||||
| 类型推断失败 | 漏报 | 标记 `LOW_CONFIDENCE`,配置 `manual_mappings` |
|
||||
| Lombok 复杂注解 | 字段遗漏 | 基于源码字段 + 注解;后续 delombok |
|
||||
| 同一 key 多分支写不同类型 | 误报 | 报告注明置信度;人工 suppression |
|
||||
| 浅克隆 parent 不可用 | 跳过检测 | `--depth 2`;文档明确要求 |
|
||||
| 一次 push 多 commit | 仅检最后一个 | 文档说明;后续扩展 range |
|
||||
| 浅克隆拿不到 before | 漏检 / exit 2 | 按 SHA `fetch --depth 1` + deepen 兜底;见 CI 说明 |
|
||||
| 一次 push 多 commit | 旧方案仅看末 commit 会漏检 | 已改为 `before..after` 累计对比 |
|
||||
| 依赖 jar 中的类型 | 字段展开不完整 | 配置 `manual_mappings` 补充 |
|
||||
| JsonUtil 实现不可见 | 序列化规则猜测 | 默认按字段名序列化;与 Fastjson 对齐 |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user