feat: 分发任务中以制品维度来记录详细日志 TencentBlueKing#1675

xujian-xj · Jan 25, 2024 · b235212 · b235212
1 parent ade04d8
commit b235212
Show file tree

Hide file tree

Showing 33 changed files with 753 additions and 272 deletions.
diff --git a/docs/apidoc/replication/record.md b/docs/apidoc/replication/record.md
@@ -1,3 +1,5 @@
+
+
 # Replication仓库同步执行日志接口
 
 [toc]
@@ -240,58 +242,62 @@
   此接口无请求体
 - 请求字段说明
 
-  |字段|类型|是否必须|默认值|说明|Description|
-  |---|---|---|---|---|---|
-  |recordId|string|是|无|任务执行日志唯一id|record id|
-  |pageNumber|int|是|无|当前页|page number|
-  |pageSize|int|是|无|分页数量|page size|
-  |packageName|string|否|无|包名称，支持前缀模糊匹配|package name|
-  |repoName|string|否|无|仓库名称|repo name|
-  |clusterName|string|否|无|远程节点名称|cluster node name|
-  |path|string|否|无|路径名称，支持前缀模糊匹配|file path|
-  |status|enum|否|无|[SUCCESS,RUNNING,FAILED]|execute status|
-
+  |字段| 类型     |是否必须|默认值| 说明                       | Description       |
+  |---|--------|---|---|--------------------------|-------------------|
+  |recordId| string |是|无| 任务执行日志唯一id               | record id         |
+  |pageNumber| int    |是|无| 当前页                      | page number       |
+  |pageSize| int    |是|无| 分页数量                     | page size         |
+  |packageName| string |否|无| 包名称，支持前缀模糊匹配             | package name      |
+  |repoName| string |否|无| 仓库名称                     | repo name         |
+  |clusterName| string |否|无| 远程节点名称                   | cluster node name |
+  |path| string |否|无| 路径名称，支持前缀模糊匹配            | file path         |
+  |status| enum   |否|无| [SUCCESS,RUNNING,FAILED] | execute status    |
+  |artifactName| string |否|无| 制品名称（Generic为fullPath） | artifact name |
+  |version| string |否|无| 版本 | artifact version |
+
+
 - 响应体
 
   ```json
   {
-    "code": 0,
-    "message": null,
-    "data": {
-      "pageNumber": 0,
-      "pageSize": 1,
-      "totalRecords": 18,
-      "totalPages": 2,
-      "records": [
-        {
-          "id": "979b573d53efcd752bf9b762",
-          "recordId": "609b573d53ccce752bf9b860",
-          "localCluster": "wuxi",
-          "remoteCluster": "wuhan",
-          "localRepoName": "npm-local",
-          "repoType": "NPM",
-          "packageConstraint": {
-            "packageKey": "npm://helloworld",
-            "versions": ["1.1.0","1.3.0"]
-          },
-          "pathConstraint": {
-            "path": "/busy/box.txt"
-          },
-          "status": "SUCCESS",
-          "progress": {
-            "success": 10,
-            "skip": 0,
-            "failed": 0,
-            "totalSize": 10
-          },
-          "startTime": "2021-05-12T12:19:08.813",
-          "endTime": "2021-05-12T12:19:37.967",
-          "errorReason": null
-        }
-      ]
-    },
-    "traceId": null
-  }
+  "code" : 0,
+  "message" : null,
+  "data" : {
+    "pageNumber" : 1,
+    "pageSize" : 20,
+    "totalRecords" : 2,
+    "totalPages" : 1,
+    "records" : [ {
+      "id" : "65b07d80b69e4404f9ba4a50",
+      "recordId" : "65b07d7eb69e4404f9ba4a4e",
+      "localCluster" : "center",
+      "remoteCluster" : "dev",
+      "localRepoName" : "generic",
+      "repoType" : "GENERIC",
+      "packageConstraint" : null,
+      "pathConstraint" : null,
+      "artifactName" : "/bkrepo.op.conf.bak",
+      "version" : null,
+      "conflictStrategy" : "SKIP",
+      "size" : 646,
+      "sha256" : "bcdf4256783b6d9cf3cb7bcd666e59c6e0a5d7e58e45c9469bb29a60a69ffff4",
+      "status" : "SUCCESS",
+      "progress" : {
+        "success" : 0,
+        "skip" : 0,
+        "failed" : 0,
+        "totalSize" : 0,
+        "conflict" : 0
+      },
+      "startTime" : "2024-01-24T11:01:20.534",
+      "endTime" : "2024-01-24T11:01:20.556",
+      "errorReason" : ""
+    } ],
+    "count" : 2,
+    "page" : 1
+  },
+  "traceId" : "2f6803fcf2b5d1ab8cbbb97fa1d20b52"
+}
   ```
 
 - data字段说明
@@ -304,19 +310,83 @@
   |remoteCluster|string|远程集群名称|remote cluster node name|
   |localRepoName|string|本地仓库名称|local repository name|
   |repoType|enum|[DOCKER,NPM,RPM,...]|local repository type|
-  |packageConstraints|object|否|无|包限制|package constraints|
-  |pathConstraints|object|否|无|路径限制|path constraints|
+  |packageConstraints|object|否|无|
+  |pathConstraints|object|否|无|
+  |artifactName|string|制品名称（历史数据为null）|artifact name|
+  |version|string|版本（Generic制品和历史数据为null）|artifact version|
+  |conflictStrategy|string|冲突策略（没有冲突和历史数据为null）|conflict strategy|
+  |size|long|制品大小|artifact size|
+  |sha256|string|制品摘要（依赖源和历史数据为null）|artifact digest|
   |status|enum|[RUNNING,SUCCESS,FAILED]|task execute status|
-  |progress|object|同步进度|task execute progress|
+  |~~progress~~|~~object~~|~~同步进度~~|~~task execute progress~~|
   |startTime|date|任务开始执行时间|task execute start time|
   |endTime|date|任务结束执行时间|task execute end time|
   |errorReason|string|错误原因，未执行或执行成功则为null|task failed error reason|
 
-- progress字段说明
+- ~~progress字段说明(废弃)~~
 
   |字段|类型|说明|Description|
   |---|---|---|---|
-  |success|long|成功数量|success size|
-  |skip|long|跳过数量|skip size|
-  |failed|long|失败数量|failed size|
-  |totalSize|long|数据总量|total size|
+  |~~success~~|~~long~~|~~成功数量~~|~~success size~~|
+  |~~skip~~|~~long~~|~~跳过数量~~|~~skip size~~|
+  |~~failed~~|~~long~~|~~失败数量~~|~~failed size~~|
+  |~~totalSize~~|~~long~~|~~数据总量~~|~~total size~~|
+
+
+
+## 根据recordId查询任务执行总览
+
+- API: GET /replication/api/task/record/overview/{recordId}
+
+- API 名称: get_task_record_overview
+
+- 功能说明：
+
+  - 中文：根据recordId查询任务执行总览
+  - English：get task record overview
+
+- 请求体
+  此接口无请求体
+
+- 请求字段说明
+
+  | 字段        | 类型   | 是否必须 | 默认值 | 说明                       | Description       |
+  | ----------- | ------ | -------- | ------ | -------------------------- | ----------------- |
+  | recordId    | string | 是       | 无     | 任务执行日志唯一id         | record id         |
+
+
+- 响应体
+
+  ```json
+  {
+      "code": 0,
+      "message": null,
+      "data": {
+          "success": 0,
+          "failed": 0,
+          "conflict": 0
+      },
+      "traceId": "6c4376c1f002127e9444d9897d0ee7ce"
+  }
+  ```
+
+- 历史数据响应体
+
+  ```json
+  {
+  	"code": 0, 
+      "message": null,
+      "data": null,
+      "traceId": "3df4fdf97a77684e352fd17973064ece"
+  }
+  ```
+
+
+
+- data字段说明(历史数据为null)
+
+  | 字段     | 类型 | 说明     | Description   |
+  | -------- | ---- | -------- | ------------- |
+  | success  | long | 成功数量 | success size  |
+  | failed   | long | 失败数量 | failed size   |
+  | conflict | long | 冲突数量 | conflict size |
diff --git a/docs/apidoc/replication/replication.md b/docs/apidoc/replication/replication.md
@@ -44,7 +44,9 @@
     },
     "remoteClusterIds": ["651095dfe0524ce9b3ab53d13532361c","329fbcda45944fb9ae5c2573acd7bd2a"],
     "enabled": true,
-    "description": "test replica task"
+    "description": "test replica task",
+    "record" : true,
+    "recordReserveDays" : 30
   }
   ```
 
@@ -61,6 +63,8 @@
   |remoteClusterIds|list|是|无|远程集成节点id|the remote cluster node ids|
   |enabled|bool|是|true|计划是否启动|do task enabled|
   |description|sting|否|无|描述|description|
+  |record|bool|是|true|是否记录分发详细日志|is record replica detail|
+  |recordReserveDays|long|是|30|分发详细日志保留天数|replica detail reserve days|
 
 - replicaTaskObjects对象说明
 
@@ -132,6 +136,8 @@
       "nextExecutionTime": "2020-03-17T12:00:00.000",
       "executionTimes": 5,
       "enabled": true,
+      "record" : true,
+      "recordReserveDays" : 30,
       "createdBy" : "system",
       "createdDate" : "2020-03-16T12:13:03.371",
       "lastModifiedBy" : "system",
@@ -195,6 +201,8 @@
       "nextExecutionTime": "2020-03-17T12:00:00.000",
       "executionTimes": 5,
       "enabled": true,
+      "record" : true,
+      "recordReserveDays" : 30,
       "createdBy" : "system",
       "createdDate" : "2020-03-16T12:13:03.371",
       "lastModifiedBy" : "system",
@@ -211,8 +219,8 @@
   |id|string|任务唯一id|task id|
   |key|string|任务唯一key|task key|
   |name|string|任务名称|task name|
-  |projectId|string/所属项目id|task projectId|
-  |replicaObjectType|enum|是|无|[REPOSITORY,PACKAGE,PATH]|replication object type|
+  |projectId|string/所属项目id|task projectId||
+  |replicaObjectType|enum|是|无|
   |replicaType|enum|[SCHEDULED,REAL_TIME]|replica type|
   |setting|object|计划相关设置|task setting|
   |remoteClusters|set|远程集成节点信息|the remote cluster node info|
@@ -221,7 +229,9 @@
   |lastExecutionTime|date|上次执行时间|task last execution time|
   |nextExecutionTime|date|下次执行时间|task next execution time|
   |executionTimes|long|执行次数|execution times|
-  |enabled|bool|是|true|计划是否启动|do task enabled|
+  |enabled|bool|是|true|
+  |record|bool|是否记录分发详细日志|is record replica detail|
+  |recordReserveDays|long|分发详细日志保留天数|replica detail reserve days|
   |createdBy|string|创建者|create user|
   |createdDate|string|创建时间|create time|
   |lastModifiedBy|string|上次修改者|last modify user|
@@ -528,7 +538,9 @@
       }
     ],
     "remoteClusterIds": ["651095dfe0524ce9b3ab53d13532361c","329fbcda45944fb9ae5c2573acd7bd2a"],
-    "description": "test replica task"
+    "description": "test replica task",
+    "record" : true,
+    "recordReserveDays" : 30
   }
   ```
 
@@ -543,6 +555,8 @@
   |replicaTaskObjects|object|是|无|同步对象信息|replication object info|
   |remoteClusterIds|list|是|无|远程集成节点id|the remote cluster node ids|
   |description|sting|否|无|描述|description|
+  |record|bool|是|true|是否记录分发详细日志|is record replica detail|
+  |recordReserveDays|long|是|30|分发详细日志保留天数|replica detail reserve days|
 
 - replicaTaskObjects对象说明
 

diff --git a/src/backend/job/biz-job/src/main/kotlin/com/tencent/bkrepo/job/Constants.kt b/src/backend/job/biz-job/src/main/kotlin/com/tencent/bkrepo/job/Constants.kt
@@ -55,7 +55,9 @@ const val DELETED_DATE = "deleted"
 const val FULL_PATH = "fullPath"
 const val PATH = "path"
 const val LAST_MODIFIED_BY = "lastModifiedBy"
-
+const val KEY = "key"
+const val RECORD_RESERVE_DAYS = "recordReserveDays"
+const val REPLICA_TYPE = "replicaType"
 /**
  * 缓存类型
  */