From bc162a7f4e23b2d7e892fa8a9ae3d987d864f771 Mon Sep 17 00:00:00 2001 From: yh263208 Date: Mon, 23 Oct 2023 12:17:09 +0800 Subject: [PATCH 1/8] docs: add changelogs for ODC 4.2.2 --- CHANGELOG-zh-CN.md | 53 ++++++++++++++++++++++++++++++++++++ CHANGELOG.rst | 68 ++++++++++++++++++++++++++++++++++++++++++++++ client | 2 +- 3 files changed, 122 insertions(+), 1 deletion(-) diff --git a/CHANGELOG-zh-CN.md b/CHANGELOG-zh-CN.md index 3860708c06..2c4d1ff04c 100644 --- a/CHANGELOG-zh-CN.md +++ b/CHANGELOG-zh-CN.md @@ -1,5 +1,58 @@ # OceanBase Developer Center (ODC) CHANGELOG +## 4.2.2 (2023-10-31) + +### 功能变化 + +OceanBase 兼容性 + +- 支持 OceanBase 4.2.1 + +基于项目的管控协同 + +- 数据源增加初始化脚本以及自定义 JDBC 连接参数设置 + +导入导出 + +- 升级 ob-loader-dumper 版本到 4.2.5-RELEASE + +数据脱敏 + +- 支持视图脱敏 + +### 缺陷修复 + +PL 调试 + +- 调试过程中无法跳入程序包中定义的子存储过程/函数 + +SQL 执行 + +- 执行 SQL 过程中持续执行 "DROP PACKAGE" 语句导致大量报错 +- 连接 OceanBase MySQL 租户时自动调用 "obodc_procedure_feature_test" 存储过程导致报错或进入连接缓慢 + +SQL-Check + +- OceanBase Oracle 租户下创建 type 时，如果子存储过程/函数无参数列表，SQL Check 报语法错误 + +数据脱敏 + +- SELECT 语句中含有多表 JOIN 的场景下脱敏失败 +- 大小写敏感的 OceanBase MySQL 模式下无法识别到敏感列导致脱敏失效 + +数据库对象管理 + +- 没有 show create view 权限用户查看视图详情时报错 +- 查看表对象时所有字符类型的长度无法显示 + +数据库变更 + +- 数据库变更任务超时时间设置无效 + +### 安全加固 + +- 支持连接密码等敏感字段的非对称加密 + ## 4.2.1 (2023-09-25) ### 缺陷修复 diff --git a/CHANGELOG.rst b/CHANGELOG.rst index ca1cba4513..ad4a25a48f 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -3,12 +3,78 @@ New ~~~ +- Feat: make odc adapt to OceanBase 4.2 (#541) [IL MARE] +- Feat(ob-sql-parser): make ob-sql-parser adapt to OceanBase 4.2 (#441) + [IL MARE] +- Feat(connection): add initialization configuration capabilities for + data sources (#488) [IL MARE] +- Feat(data-transfer): upgrade ob-loader-dumper to 4.2.5-RELEASE (#494) + [LuckyLeo] +- Feat(integration): support retrieve xml format response (#338) + [XiaoYang] +- Feat(data-security): data masking support columns in view (#97) + [XiaoYang] +- Feat(encryption): support asymmetric encryption (#99) [XiaoYang] +- Feat(schema-plugin): schema-plugin access service layer (#88) + [zhangxiao] + +Changes +~~~~~~~ +- Refactor(unit-test): cherry-pick unit-test commits from 4.2.x to 4.2.2 + (#474) [XiaoYang] +- Refactor(submodule): update submodule (#470) [IL MARE] +- Refactor(unit-test): refact unit test cases (#139) (#142) [IL MARE] +- Refactor(ob-sql-parser): add several new syntaxes which added in + OceanBase 4.1.0 (#132) [IL MARE] +- Refactor(unit-test): refact unit test cases (#139) (#141) [IL MARE] + +Fix +~~~ +- Fix(schema-plugin): cannot display constraint name for ob oralce 4.2.1 + (#533) [zhangxiao] +- Fix(pl-debug): fix failed to step in a subprocedure or subfunction + defined in package (#566) [IL MARE] +- Fix(integration): recover bastion integration (#559) [yizhou] +- Fix(databasechange): fix task costs too much time to start up (#551) + [IL MARE] +- Fix: remove pl delete code (#548) [IL MARE] +- Fix(ob-sql-parser): fix failed to parse member proc without parameters + (#546) [IL MARE] +- Fix(osc): fix get cloud main account id throw exception when + environment is not cloud (#529) [krihy] +- Fix(data-security): exist sensitive is not filtered and view + desensitization data failed (#509) [XiaoYang] +- Fix(unit-test): unit test logs expose sensitive information (#498) + (#516) [XiaoYang] +- Fix(view): fix get view failed without show view permission (#507) + [zhangxiao] +- Fix: masking failed (#485) [XiaoYang] +- Fix(osc): execute pre and post interceptor in retry rename table + (#486) [krihy] +- Fix(unit-test): fix failed unit test cases (#476) [XiaoYang, yh263208] +- Fix(data-security): error metadata of built-in sensitive algorithm + (#458) [XiaoYang] +- Fix: database change failed (#455) [XiaoYang] +- Fix: scan sensitive columns (#444) [XiaoYang] +- Fix(mvc): api response content type converts to xml (#377) [XiaoYang] +- Fix: extract column from SQL with multiple join clauses (#327) + [XiaoYang] + + +v4.2.1 (2023-10-09) +------------------- + +New +~~~ +- Feat(db-browser): upgrade db-browser's version to 1.0.2 (#402) [IL + MARE] - Feat(data-transfer): support saving export objects (#73) [LuckyLeo] - Feat(workflow): add checkbox for installing db-browser and ob-sql- parser (#75) [IL MARE] Changes ~~~~~~~ +- Refactor(submodule): update submodule (#436) [IL MARE] - Refactor(migration): extract data migration interface (#290) [pynzzZ] - Refactor(migrates): add some abstract methods for migrates (#275) [IL MARE] @@ -24,6 +90,8 @@ Changes Fix ~~~ +- Fix(dlm): validate condition by sql explain. (#440) [guowl3] +- Fix(datasource): optimize datasource synchronization (#391) [pynzzZ] - Fix(osc): osc support ob ce add type ob mysql ce (#390) [krihy] - Fix: masking enabled (#383) [XiaoYang] - Fix(clientMode): fail to start for lack of Service annotations (#371) diff --git a/client b/client index 91dabffcdc..e98b448aa7 160000 --- a/client +++ b/client @@ -1 +1 @@ -Subproject commit 91dabffcdc65bf8a9c9091c411337e61977ce526 +Subproject commit e98b448aa763dfb0321d9bc1e1dea6cea362fd41 From 95c218cf85e4a5ad2b0ddaab69f35ab2bce4526b Mon Sep 17 00:00:00 2001 From: yh263208 Date: Mon, 23 Oct 2023 13:56:25 +0800 Subject: [PATCH 2/8] reponse to cr comments --- CHANGELOG-zh-CN.md | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/CHANGELOG-zh-CN.md b/CHANGELOG-zh-CN.md index 2c4d1ff04c..4263c1b45d 100644 --- a/CHANGELOG-zh-CN.md +++ b/CHANGELOG-zh-CN.md @@ -4,22 +4,21 @@ ### 功能变化 -OceanBase 兼容性 - -- 支持 OceanBase 4.2.1 - -基于项目的管控协同 +数据源 +- 支持 MySQL 数据源 +- 适配 OceanBase 4.2.0/4.2.1 - 数据源增加初始化脚本以及自定义 JDBC 连接参数设置 -导入导出 - -- 升级 ob-loader-dumper 版本到 4.2.5-RELEASE - 数据脱敏 - 支持视图脱敏 +数据归档 + +- 支持 OceanBase 4.x 版本 +- 支持 MySQL 到 OceanBase 链路 + ### 缺陷修复 PL 调试 @@ -49,9 +48,14 @@ SQL-Check - 数据库变更任务超时时间设置无效 +### 依赖库升级 + +- 升级 ob-loader-dumper 版本到 4.2.5-RELEASE +- 升级 oceanbase-client 版本到 2.4.5 + ### 安全加固 -- 支持连接密码等敏感字段的非对称加密 +- 前后端敏感字段传输使用非对称加密 ## 4.2.1 (2023-09-25) From 2dac1b6b401d9b8522a55a4ae87b3ecfb9bba112 Mon Sep 17 00:00:00 2001 From: yh263208 Date: Mon, 23 Oct 2023 16:26:05 +0800 Subject: [PATCH 3/8] reponse to cr comments --- CHANGELOG-zh-CN.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/CHANGELOG-zh-CN.md b/CHANGELOG-zh-CN.md index 4263c1b45d..8ccf2e69e7 100644 --- a/CHANGELOG-zh-CN.md +++ b/CHANGELOG-zh-CN.md @@ -19,6 +19,13 @@ - 支持 OceanBase 4.x 版本 - 支持 MySQL 到 OceanBase 链路 +### 改进 + +- 优化了大规模表列场景下的对象管理性能，通过 ob-sql-parser 解析索引/约束的元数据 +- 优化了数据库对象树交互，项目和数据源的选择交互区域折叠到顶部，数据库列表展示更清晰 +- 优化了 SQL 窗口新建和 SQL 窗口切换数据库的交互，切换数据库更快速了，SQL 窗口增加复制操作 +- 优化了数据脱敏配置交互，选择敏感列更方便 + ### 缺陷修复 PL 调试 From 36ed7e61aa3405cc65f1351cdea4a6b7e63c7d21 Mon Sep 17 00:00:00 2001 From: yh263208 Date: Mon, 23 Oct 2023 17:15:51 +0800 Subject: [PATCH 4/8] builds: re-run workflows --- CHANGELOG-zh-CN.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG-zh-CN.md b/CHANGELOG-zh-CN.md index 8ccf2e69e7..6b6bd9b4c2 100644 --- a/CHANGELOG-zh-CN.md +++ b/CHANGELOG-zh-CN.md @@ -1,6 +1,6 @@ # OceanBase Developer Center (ODC) CHANGELOG -## 4.2.2 (2023-10-31) +## 4.2.2 (2023-10-30) ### 功能变化 From 5fc26ac40ab5352db4afbcf7c84e14c3a3a9516f Mon Sep 17 00:00:00 2001 From: yh263208 Date: Tue, 31 Oct 2023 14:51:54 +0800 Subject: [PATCH 5/8] docs(changelog): add new changes to changelog --- CHANGELOG-zh-CN.md | 17 +++++++++- CHANGELOG.md | 79 ++++++++++++++++++++++++++++++++++++++++++++++ CHANGELOG.rst | 30 ++++++++++++++++++ 3 files changed, 125 insertions(+), 1 deletion(-) diff --git a/CHANGELOG-zh-CN.md b/CHANGELOG-zh-CN.md index 6b6bd9b4c2..7f2ff2d9a2 100644 --- a/CHANGELOG-zh-CN.md +++ b/CHANGELOG-zh-CN.md @@ -1,6 +1,6 @@ # OceanBase Developer Center (ODC) CHANGELOG -## 4.2.2 (2023-10-30) +## 4.2.2 (2023-11-02) ### 功能变化 @@ -18,6 +18,7 @@ - 支持 OceanBase 4.x 版本 - 支持 MySQL 到 OceanBase 链路 +- 支持白屏限流配置 ### 改进 @@ -25,6 +26,8 @@ - 优化了数据库对象树交互，项目和数据源的选择交互区域折叠到顶部，数据库列表展示更清晰 - 优化了 SQL 窗口新建和 SQL 窗口切换数据库的交互，切换数据库更快速了，SQL 窗口增加复制操作 - 优化了数据脱敏配置交互，选择敏感列更方便 +- 优化了存在大量数据源场景下获取数据源列表缓慢以及获取数据源状态缓慢的问题 +- 优化以错误参数运行 PL 运行时的报错文案 ### 缺陷修复 @@ -36,10 +39,13 @@ SQL 执行 - 执行 SQL 过程中持续执行 "DROP PACKAGE" 语句导致大量报错 - 连接 OceanBase MySQL 租户时自动调用 "obodc_procedure_feature_test" 存储过程导致报错或进入连接缓慢 +- SQL 执行耗时统计各子项耗时之和不等于父项 +- SQL 执行耗时统计中， "SQL 预检查"及"SQL 后置检查"缺乏详细子项耗时统计 SQL-Check - OceanBase Oracle 租户下创建 type 时，如果子存储过程/函数无参数列表，SQL Check 报语法错误 +- 无法关闭"语法错误"规则 数据脱敏 @@ -55,6 +61,15 @@ SQL-Check - 数据库变更任务超时时间设置无效 +导入导出 + +- Oracle 模式下导出程序包不包含包体 +- 无法将含有制表符的结果集导出为 Excel + +操作审计 + +- 没有将"SQL 检查规范"以及"SQL 窗口规范"改变纳入操作审计范围 + ### 依赖库升级 - 升级 ob-loader-dumper 版本到 4.2.5-RELEASE diff --git a/CHANGELOG.md b/CHANGELOG.md index aa78001e33..f5a52322ec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,84 @@ # OceanBase Developer Center (ODC) CHANGELOG +## 4.2.2 (2023-11-02) + +### Feature + +Data Source + +- Support MySQL data source +- Adapted to OceanBase 4.2.0/4.2.1 +- Data sources add initialization scripts and customized JDBC connection parameter settings + +Data Desensitization + +- Support view desensitization + +DLM + +- Support OceanBase 4.x version +- Support MySQL to OceanBase link +- Support white screen current limiting configuration + +### Improve + +- Optimized object management performance in large-scale table scenarios, and used ob-sql-parser to parse index/constraint metadata +- Optimized database object tree interaction, the project and data source selection interaction area is folded to the top, and the database list is displayed more clearly +- Optimized the interaction between creating a new SQL window and switching databases in the SQL window. Switching databases is faster, and the SQL window adds a copy operation. +- Optimized the data desensitization configuration interaction, making it more convenient to select sensitive columns +- Optimized the problem of slow retrieval of data source list and slow retrieval of data source status in scenarios where there are a large number of data sources. +- Optimize the error text when running PL with wrong parameters + +### Fix + +PL Debugging + +- Unable to jump into sub-stored procedures/functions defined in the package during debugging + +SQL Execution + +- Continuous execution of "DROP PACKAGE" statements during SQL execution resulted in a large number of error reports +- When connecting to the OceanBase MySQL tenant, the "obodc_procedure_feature_test" stored procedure is automatically called, resulting in an error or slow connection. +- The sum of the time consumption of each sub-item in SQL execution time-consuming statistics is not equal to the parent item +- In the SQL execution time-consuming statistics, "SQL pre-check" and "SQL post-check" lack detailed sub-item time-consuming statistics + +SQL-Check + +- When creating type under the OceanBase Oracle tenant, if the sub-stored procedure/function has no parameter list, SQL Check will report a syntax error. +- Unable to turn off "Syntax Error" rule + +Data Desensitization + +- Desensitization fails when the SELECT statement contains multiple table JOINs +- Sensitive columns cannot be recognized in the case-sensitive OceanBase MySQL mode, causing desensitization to fail. + +Database Object Management + +- Users without the show create view permission will receive an error when viewing view details. +- The length of all character types cannot be displayed when viewing table objects + +Database Changes + +- The database change task timeout setting is invalid + +Import and Export + +- The exported package in Oracle mode does not contain the package body +- Unable to export result set containing tab characters to Excel + +Operational audit + +- Changes to "SQL Check Specification" and "SQL Window Specification" are not included in the operational audit scope + +### Dependency library upgrade + +- Upgrade ob-loader-dumper version to 4.2.5-RELEASE +- Upgrade oceanbase-client version to 2.4.5 + +### Security reinforcement + +- Transmission of sensitive fields on the front and back ends uses asymmetric encryption + ## 4.2.1 (2023-09-25) ### Fix diff --git a/CHANGELOG.rst b/CHANGELOG.rst index ad4a25a48f..b2dba1d0d7 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -3,6 +3,11 @@ New ~~~ +- Feat(dlm):support configuring limiter (#626) [guowl3] +- Feat(data-security): add data type unit into response (#629) + [XiaoYang] +- Feat(dlm): data archive supports MySQL to OB (#544) [guowl3] +- Feat: add timeout settings for pl-debug (#576) [IL MARE] - Feat: make odc adapt to OceanBase 4.2 (#541) [IL MARE] - Feat(ob-sql-parser): make ob-sql-parser adapt to OceanBase 4.2 (#441) [IL MARE] @@ -30,6 +35,31 @@ Changes Fix ~~~ +- Fix: dirty meta data (#663) [XiaoYang] +- Fix(sql-execute): fix failed to get time consuming (#658) [IL MARE] +- Fix(migration): rule metadata migration will be triggered every time + the ODC server starts up (#649) [pynzzZ] +- Fix(sql-check): fix syntax error check rule can not be disabled (#652) + [IL MARE] +- Fix: fix can not get plan (#660) [IL MARE] +- Fix(data-transfer): no package body (#653) [LuckyLeo] +- Fix(web): editor.worker.js static resource 404 not found (#656) + [pynzzZ] +- Fix(data-transfer): fix wrong data objects and schema objects (#620) + [LuckyLeo] +- Fix(datasource): the data source list refreshes very slowly and cannot + obtain the connect status while there are a huge amount of data + sources (#599) [pynzzZ, yh263208] +- Fix: fix failed to query data and sql rules changing is not recorded + by audit event (#608) [IL MARE] +- Fix(connection): fix failed to set setConnectionAttrs (#601) [IL MARE] +- Fix(db-browser): cannot get table charset in native mysql mode (#592) + [zhangxiao] +- Fix(result-export): failed to convert CSV file into Excel file (#586) + [LuckyLeo] +- Fix(diagnose): optimize log information when explain failed (#589) + [LuckyLeo] +- Fix(pl): fix wrong parameter check error message (#583) [IL MARE] - Fix(schema-plugin): cannot display constraint name for ob oralce 4.2.1 (#533) [zhangxiao] - Fix(pl-debug): fix failed to step in a subprocedure or subfunction From 1776ce1357ccb0407281c6b25b86755c26a25990 Mon Sep 17 00:00:00 2001 From: yh263208 Date: Tue, 31 Oct 2023 14:54:36 +0800 Subject: [PATCH 6/8] docs(changelog): add new changes to changelog --- CHANGELOG-zh-CN.md | 1 + CHANGELOG.md | 1 + 2 files changed, 2 insertions(+) diff --git a/CHANGELOG-zh-CN.md b/CHANGELOG-zh-CN.md index 7f2ff2d9a2..a19621cd22 100644 --- a/CHANGELOG-zh-CN.md +++ b/CHANGELOG-zh-CN.md @@ -60,6 +60,7 @@ SQL-Check 数据库变更 - 数据库变更任务超时时间设置无效 +- 个人空间下查看定时执行的带有自动回滚节点的数据库变更任务详情失败 导入导出 diff --git a/CHANGELOG.md b/CHANGELOG.md index f5a52322ec..17e4fb7eeb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -56,6 +56,7 @@ Database Object Management - Users without the show create view permission will receive an error when viewing view details. - The length of all character types cannot be displayed when viewing table objects +- Failed to view details of scheduled database change tasks with automatic rollback nodes in personal space Database Changes From 6a6886dbba8f4eeab8ffa9bb6bfdee4559787ab2 Mon Sep 17 00:00:00 2001 From: yh263208 Date: Wed, 1 Nov 2023 14:37:25 +0800 Subject: [PATCH 7/8] docs: update changelog --- CHANGELOG-zh-CN.md | 5 +++++ CHANGELOG.md | 5 +++++ CHANGELOG.rst | 8 ++++++++ 3 files changed, 18 insertions(+) diff --git a/CHANGELOG-zh-CN.md b/CHANGELOG-zh-CN.md index a19621cd22..08389fac1e 100644 --- a/CHANGELOG-zh-CN.md +++ b/CHANGELOG-zh-CN.md @@ -19,6 +19,11 @@ - 支持 OceanBase 4.x 版本 - 支持 MySQL 到 OceanBase 链路 - 支持白屏限流配置 +- 支持断点恢复 + +导入导出 + +- 支持类型对象的导入导出 ### 改进 diff --git a/CHANGELOG.md b/CHANGELOG.md index 17e4fb7eeb..c541dd24d5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,11 @@ DLM - Support OceanBase 4.x version - Support MySQL to OceanBase link - Support white screen current limiting configuration +- Support breakpoint recovery + +Import and Export + +- Supports import and export of type objects ### Improve diff --git a/CHANGELOG.rst b/CHANGELOG.rst index b2dba1d0d7..1535807d53 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -3,6 +3,7 @@ New ~~~ +- Feat(dlm):support breakpoint recovery (#635) [guowl3] - Feat(dlm):support configuring limiter (#626) [guowl3] - Feat(data-security): add data type unit into response (#629) [XiaoYang] @@ -35,6 +36,13 @@ Changes Fix ~~~ +- Fix(dlm): submit task got condition not supported error while + condition contains subquery (#668) [guowl3] +- Fix(database-change): failed to view a scheduled database change task + with rollback plan in personal space (#669) [zhangxiao] +- Fix(pl-debug): enable dbms_output first (#677) [IL MARE] +- Fix(database): use datasource's environment as database's environment + to prevent data inconsistency (#659) [pynzzZ] - Fix: dirty meta data (#663) [XiaoYang] - Fix(sql-execute): fix failed to get time consuming (#658) [IL MARE] - Fix(migration): rule metadata migration will be triggered every time From 5ad1773ff510c89717c04323dc1c9a39b405efe3 Mon Sep 17 00:00:00 2001 From: yh263208 Date: Wed, 1 Nov 2023 14:40:05 +0800 Subject: [PATCH 8/8] docs: update changelog --- CHANGELOG-zh-CN.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG-zh-CN.md b/CHANGELOG-zh-CN.md index 08389fac1e..cddc75a6dc 100644 --- a/CHANGELOG-zh-CN.md +++ b/CHANGELOG-zh-CN.md @@ -32,7 +32,7 @@ - 优化了 SQL 窗口新建和 SQL 窗口切换数据库的交互，切换数据库更快速了，SQL 窗口增加复制操作 - 优化了数据脱敏配置交互，选择敏感列更方便 - 优化了存在大量数据源场景下获取数据源列表缓慢以及获取数据源状态缓慢的问题 -- 优化以错误参数运行 PL 运行时的报错文案 +- 优化以错误参数运行 PL 时的报错文案 ### 缺陷修复