Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

0.3.1 release #115

Merged
merged 1,602 commits into from
Jun 18, 2024
Merged

0.3.1 release #115

merged 1,602 commits into from
Jun 18, 2024

Conversation

Alexkun
Copy link
Collaborator

@Alexkun Alexkun commented Jun 5, 2024

What is the purpose of the change

Related issues: #95, #96, #97, #98, #99, #100, #101, #102, #103, #104, #105, #106, #107, #108, #109, #110, #111, #112, #113, #114

Brief change log

  1. Support job-level resource downloading.
  2. Supports viewing and downloading of meta.json of existing execution history tasks.
  3. Add audit log recording to Streamis.
  4. Support batch enable and disable jobs.
  5. Adjust the default snapshot root address based on the high availability policy.
  6. Add cluster high availability query interface.
  7. Add heartbeat for callback log.
  8. add hook before flink apps stop.
  9. Add token in /upload interface.
  10. Add support for args with spaces.
  11. Error code optimization to identify failed submission exceptions
  12. Adding project-level configuration templates.

Verifying this change

(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

  • Added tests for creating and execute the Streamis jobs and verify the availability of different Streamis Job, such as flinkSQL, Jar.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (yes / no)
  • Anything that affects deployment: (yes / no / don't know)
  • The Core framework, i.e., JobManager, Server.: (yes / no)

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

Alexkun and others added 30 commits January 7, 2024 17:38
bertcai and others added 25 commits April 30, 2024 09:54
…dev-0.3.8-webank

# Conflicts:
#	streamis-jobmanager/streamis-job-launcher/streamis-job-launcher-base/src/main/scala/com/webank/wedatasphere/streamis/jobmanager/launcher/job/conf/JobConf.scala
2. refactor msg in hook execution
2. update linkis version to '1.5.0'
2. update linkis version to '1.5.0'
3. refactor pom.xml
Copy link
Collaborator

@Davidhua1996 Davidhua1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@Alexkun Alexkun merged commit ffa724e into WeBankFinTech:main Jun 18, 2024
2 checks passed
@Alexkun Alexkun deleted the dev-0.3.1.release branch December 2, 2024 07:13
@Alexkun Alexkun restored the dev-0.3.1.release branch December 2, 2024 07:20
Alexkun added a commit that referenced this pull request Dec 27, 2024
* 1. add stream.log.enable in log-collector

* remove unused file.

* refactor JobLogRestfulApi, add lock for updating register info

* 告警信息查询支持分页

* upd: 告警表格接口与交互改为分页展示

* fix not alarm bug

* upd: 执行历史表格接口与交互改为分页展示

* 作业历史信息查询支持分页

* fix: 表格loading持续存在异常修复

* jobName空判断,长度最长支持2000

* Revert "upd: 执行历史表格接口与交互改为分页展示"

* add support for skipping uris in job-audit-log module

* fix comments in linkis.properties

* refactor StreamisHeartbeatHandler

* add ddl for table 'linkis_stream_register_info'

* update version from '0.3.6-wedatasphere-SNAPSHOT' to '0.3.7-wedatasphere-SNAPSHOT'

* fix bug in RegularUtil

* upd: 执行历史表格接口与交互改为分页展示

* upd: 支持现有执行历史任务metajson查看与下载,接口待联调

* add cost time for audit log.

* add metajson to StreamTask on job starting

* fix val bug

* fix jobconfig bug

* upd: 支持job级资源下载,接口待联调

* upd: 优化启动确认检查逻辑,增加批量启动、停止相关交互

* job materials download

* fix job materials download

* add product name

* fix path name problem

* fix job material download

* upd: 支持job级资源下载,接口联调

* add error codes solutions

* change 'contains' to 'equals' in AuditLogAspect

* Revert "change 'contains' to 'equals' in AuditLogAspect"

This reverts commit 2b979b9.

* change 'contains' to 'equals' in AuditLogAspect

* fix some problems in material download

* fix root path

* fix execute statement

* add costTimeMills in StreamAuditLogMapper.xml

* add field 'job_startup_config' in linkis_stream_task upgrade_dml.sql

* rename field 'job_startup_config' to 'job_start_config' in table linkis_stream_task

* add log in DefaultStreamTaskService

* upd: 审计日志信息增加接口耗时

* fix: metajson 弹窗文案修改

* fix error code and delete linkis error code

* fix error code statement

* fix error code solution in task table

* change root path

* fix execute problem

* upd: 执行历史任务metajson查看与下载接口联调

* fix root path

* fix problems in root path

* fix: metajson 弹窗文案修改

* errorCode solution optimization

* upd: 任务列表增加解决方案跳转链接

* product param optimization

* upd: 批量停止弹窗footer按钮与批量启动保持统一

* upd: 批量启动失败时不显示任务完成弹窗

* fix error code match

* fix String interception

* fix some problems

* fix error code default comments

* 修改配置名称

* fix: 修改baseurl,解决环境异常

* remove fields 'metainfo' and 'worksapcename' in josn of MetaJsonInfo

* fix error code default comments

* fix error code default comments

* fix metajson

* fix metajson

* add flink manager ec init extra params

* fix value

* fix AbstractLinkisJobStateFetcher

* set productName cache

* fix error msg

* upd: 知识库URL改为文件配置式

* fix: 任务列表:重启操作检查失败时关闭所有浮窗而不是仅关闭检查浮窗导致卡死

* fix: 任务列表:批量修改标签后清空选中列表

* fix high available inspect

* 1. remove unused resources.
2. change application.yml to application.properties in streamis-server

* exclude snakeyaml in streamis-server and appconn

* 1. remove unused resources.

* 1. remove unused yml resources.

* 1. exclude yml resources.

* upd: 任务列表:增加对上传的控制

* fix: 批量停止成功后隐藏批量停止弹窗

* update version from '0.3.7-wedatasphere-SNAPSHOT' to '0.3.8-wedatasphere-SNAPSHOT'

* add hook interface.

* add gitignore

* add token authentication for job upload

* add token authentication for project files upload

* add token authentication switch

* fix npe in map put

* upd: 添加水印

* upd: 水印出现页面调整

* add job template upload

* upd: streamis子页面获取用户相关base信息存入localstorage

* job template-change jobContent

* job template-upload jobContent

* upd: job配置与args参数是否支持编辑与上传逻辑一致由配置文件配置

* fix job template-upload

* fix job template-upload

* fix job template upload

* fix job template upload in projectServer

* add template in jobConf

* add template in jobContent

* fix template in jobContent

* add param separates

* fix job template in jobContent

* add interface for getting username

* add JobShutdownHook and DefaultEmptyJobShutdownHook

* change type of timeoutMills from 'int' to 'long'

* fix job template in job config

* fix npe as project does not have a template

* add jsonUtil

* add download interface for job template

* change task param

* upd: 接入用户名接口,水印颜色加深

* upd: job 任务详情增加项目模板查看与下载

* add job shutdown hook

* fix Object

* add configuration in jobContent and jobConfig

* upd: joblist增加是否弹出跳过hook异常弹窗相关逻辑

* upd: jobconfig、detail模板编辑权限调整,配置内容实用模板与自定义并集结果

* upd: job 任务详情模板调整

* chore: 更新依赖与配置文件

* fix jobTemplate download

* register default hook 'emptyJobShutdownHook'

* change registering default hook 'emptyJobShutdownHook'

* fix error code match

* upd: 调整joblist跳过hook异常逻辑

* fix: 解决editable获取的参数错误问题

* change wait radix on executing the hook

* fix: 上传接口content-type设置调整

* fix: joblist 停止poptip改为挂载到body解决被table遮断的问题

* fix: 执行弹窗关闭时同步关闭停止信息弹窗

* fix: 字段错误修复

* fix: hasHook接口字段类型变更为数组,同步更新判断TrueFalse逻辑

* refactor executeJobShutdownHook

* upd: jardetail 项目模板增加为null时的处理

* fix some problems

* fix some problems

* refactor executeJobShutdownHook

* fix job template problems

* fix jetty-server

* upd: jobconfig增加模板与job标记

* upd: jobdetail 模板下载改为前端下载

* fix job shutdown hook

* Revert "fix jetty-server"

This reverts commit c709c3b.

* fix: jobconfig 模板字段问题修复

* fix job shutdown hook

* fix job template

* fix job template

* fix job shutdown hook

* remove cache pool in DefaultStreamTaskService

* fix: jobconfig 模版字段 问题修复

* upd: flinks参数添加重复相关处理逻辑,job不可重复,template可和job重复

* fix job template

* fix:jobconfig 保存阶段自定义参数判重逻辑去除

* fix error code match

* change the type of flink separate

* fix 0.3.8 code review

* 1. add cancel in StreamisJobShutdownHook
2. refactor msg in hook execution

* fix 0.3.8 code review

* fix 0.3.8 code review

* fix 0.3.8 code review

* fix 0.3.8 code review

* add flink yarn log switch and log

* 1. update version to '0.3.1-SNAPSHOT'
2. update linkis version to '1.5.0'

* 1. update version to '0.3.1'
2. update linkis version to '1.5.0'
3. refactor pom.xml

* fix dss version

* update docs

* update sql

---------

Co-authored-by: gelxiogong <15927047952@163.com>
Co-authored-by: v_nikeyang <1013195908@qq.com>
Co-authored-by: bert_cai <bert_cai@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants