[DSIP-7] [API] Suggest refactor the backend api #10257

SbloodyS · 2022-05-26T13:44:03Z

Search before asking

I had searched in the issues and found no similar feature requirement.

Description

Purpose:
A user-friendly and high-performance restful API is our goal.

Refactor reason:

Use restful style to unify API specifications
Solve the current incomplete API document and reduce the threshold of API use
Improve API reusability
Reduce unnecessary data return and improve performance
Improve the readability of the code and reduce the maintenance cost
Improve the accuracy of error reporting

API output Standard:

Content-Type using application/json(segmented upload file is not applicable)
Users can simply understand how to call the API normally by reading the API docs
The input and output parameters of API should avoid using types that cannot be intuitively understood, such as JSON and MAP
All enumeration values in the API need to be included in the API document
Comply with restful specifications
The error information requested by API can accurately express the cause of the error
Rename all process to workflow as we discussed in [DSIP-7] [API] Suggest refactor the backend api #10257 (comment)
The service refactoring part and the controller refactoring part should be divided into two PR base on the conclusions in [Feature-10983] [refactor] ResourcesController #10998 (comment)

Please reply the controller name and its function name you want to participate in and the issue you created. I will link it to this issue and assign it to you. Anyone who wants to participate is very welcome ^_^

Current API

Login page

Controller	Function	issue	status
LoginController	login	#10459
LoginController	signOut	#10459

Project manage page

Project

Controller	Function	issue	status
ProjectController	createProject	#10489	✅
ProjectController	updateProject	#10489	✅
ProjectController	queryProjectByCode	#10489	✅
ProjectController	queryProjectListPaging	#10489	✅
ProjectController	deleteProject	#10489	✅
ProjectController	queryUnauthorizedProject	#10489	✅
ProjectController	queryAuthorizedProject	#10489	✅
ProjectController	queryAuthorizedUser	#10489	✅
ProjectController	queryProjectCreatedAndAuthorizedByUser	#10489	✅
ProjectController	queryAllProjectList	#10489	✅

Workflow Monitor

Controller	Function	issue
DataAnalysisController	countTaskState	#12969
DataAnalysisController	countProcessInstanceState	#12969
DataAnalysisController	countDefinitionByUser	#12969
DataAnalysisController	countCommandState	#12969
DataAnalysisController	countQueueState	#12969

Workflow Relationship

Controller	Function	issue	status
WorkFlowLineageController	queryWorkFlowLineageByName
WorkFlowLineageController	queryWorkFlowLineageByCode
WorkFlowLineageController	queryWorkFlowLineage
WorkFlowLineageController	verifyTaskCanDelete

Workflow Definition

Controller	Function	issue
ProcessDefinitionController	createProcessDefinition	#11129
ProcessDefinitionController	copyProcessDefinition	#11129
ProcessDefinitionController	moveProcessDefinition	#11129
ProcessDefinitionController	verifyProcessDefinitionName	#11129
ProcessDefinitionController	updateProcessDefinition	#11129
ProcessDefinitionController	queryProcessDefinitionVersions	#11129
ProcessDefinitionController	switchProcessDefinitionVersion	#11129
ProcessDefinitionController	deleteProcessDefinitionVersion	#11129
ProcessDefinitionController	releaseProcessDefinition	#11129
ProcessDefinitionController	queryProcessDefinitionByCode	#11129
ProcessDefinitionController	queryProcessDefinitionByName	#11129
ProcessDefinitionController	queryProcessDefinitionList	#11129
ProcessDefinitionController	queryProcessDefinitionSimpleList	#11129
ProcessDefinitionController	queryProcessDefinitionListPaging	#11129
ProcessDefinitionController	viewTree	#11129
ProcessDefinitionController	getNodeListByDefinitionCode	#11129
ProcessDefinitionController	getNodeListMapByDefinitionCodes	#11129
ProcessDefinitionController	getProcessListByProjectCodes	#11129
ProcessDefinitionController	getTaskListByProcessDefinitionCode	#11129
ProcessDefinitionController	deleteProcessDefinitionByCode	#11129
ProcessDefinitionController	batchDeleteProcessDefinitionByCodes	#11129
ProcessDefinitionController	batchExportProcessDefinitionByCodes	#11129
ProcessDefinitionController	queryAllProcessDefinitionByProjectCode	#11129
ProcessDefinitionController	importProcessDefinition	#11129
ProcessDefinitionController	createEmptyProcessDefinition	#11129
ProcessDefinitionController	updateProcessDefinitionBasicInfo	#11129
ProcessDefinitionController	releaseWorkflowAndSchedule	#11129

Workflow Instance

Controller	Function	issue
ProcessInstanceController	queryProcessInstanceList	#10263
ProcessInstanceController	queryTaskListByProcessId	#10263
ProcessInstanceController	updateProcessInstance	#10263
ProcessInstanceController	queryProcessInstanceById	#10263
ProcessInstanceController	queryTopNLongestRunningProcessInstance	#10263
ProcessInstanceController	deleteProcessInstanceById	#10263
ProcessInstanceController	querySubProcessInstanceByTaskId	#10263
ProcessInstanceController	queryParentInstanceBySubId	#10263
ProcessInstanceController	viewVariables	#10263
ProcessInstanceController	viewTree	#10263
ProcessInstanceController	batchDeleteProcessInstanceByIds	#10263

Task Definition

Controller	Function	issue	status
TaskDefinitionController	createTaskDefinition
TaskDefinitionController	createTaskBindsWorkFlow
TaskDefinitionController	updateTaskDefinition
TaskDefinitionController	updateTaskWithUpstream
TaskDefinitionController	queryTaskDefinitionVersions
TaskDefinitionController	switchTaskDefinitionVersion
TaskDefinitionController	deleteTaskDefinitionVersion
TaskDefinitionController	deleteTaskDefinitionByCode
TaskDefinitionController	queryTaskDefinitionDetail
TaskDefinitionController	queryTaskDefinitionListPaging
TaskDefinitionController	genTaskCodeList
TaskDefinitionController	releaseTaskDefinition

Task Instance

Controller	Function	issue	status
TaskInstanceController	queryTaskListPaging	#11713	✅
TaskInstanceController	forceTaskSuccess	#11713	✅

Task Group

Controller	Function	issue	status
TaskGroupController	createTaskGroup
TaskGroupController	updateTaskGroup
TaskGroupController	queryAllTaskGroup
TaskGroupController	queryTaskGroupByStatus
TaskGroupController	queryTaskGroupByCode
TaskGroupController	closeTaskGroup
TaskGroupController	startTaskGroup
TaskGroupController	forceStart
TaskGroupController	modifyPriority
TaskGroupController	queryTasksByGroupId

Mornitor center

Controller	Function	issue
MonitorController	listMaster	#10458
MonitorController	listWorker	#10458
MonitorController	queryDatabaseState	#10458

Datasource center

Controller	Function	issue
DataSourceController	createDataSource	#10460
DataSourceController	updateDataSource	#10460
DataSourceController	queryDataSource	#10460
DataSourceController	queryDataSourceListPaging	#10460
DataSourceController	connectDataSource	#10460
DataSourceController	connectionTest	#10460
DataSourceController	deleteDataSource	#10460
DataSourceController	verifyDataSourceName	#10460
DataSourceController	unauthDatasource	#10460
DataSourceController	authedDatasource	#10460
DataSourceController	getKerberosStartupState	#10460
DataSourceController	getTables	#10460
DataSourceController	getTableColumns	#10460

Resource center

Controller	Function	issue
ResourcesController	createDirectory	#10983
ResourcesController	createResource	#10983
ResourcesController	updateResource	#10983
ResourcesController	queryResourceList	#10983
ResourcesController	queryResourceListPaging	#10983
ResourcesController	deleteResource	#10983
ResourcesController	verifyResourceName	#10983
ResourcesController	queryResourceJarList	#10983
ResourcesController	queryResource	#10983
ResourcesController	viewResource	#10983
ResourcesController	onlineCreateResource	#10983
ResourcesController	updateResourceContent	#10983
ResourcesController	downloadResource	#10983
ResourcesController	createUdfFunc	#10983
ResourcesController	viewUIUdfFunction	#10983
ResourcesController	updateUdfFunc	#10983
ResourcesController	queryUdfFuncListPaging	#10983
ResourcesController	queryUdfFuncList	#10983
ResourcesController	verifyUdfFuncName	#10983
ResourcesController	deleteUdfFunc	#10983
ResourcesController	authorizedFile	#10983
ResourcesController	authorizeResourceTree	#10983
ResourcesController	unauthUDFFunc	#10983
ResourcesController	authorizedUDFFunction	#10983
ResourcesController	queryResourceById	#10983

TaskGroupController	createTaskGroup
TaskGroupController	updateTaskGroup
TaskGroupController	queryAllTaskGroup
TaskGroupController	queryTaskGroupByStatus
TaskGroupController	queryTaskGroupByCode
TaskGroupController	closeTaskGroup
TaskGroupController	startTaskGroup
TaskGroupController	forceStart
TaskGroupController	queryTasksByGroupId

Dataquality center

Controller	Function	issue	status
DataQualityController	getRuleFormCreateJsonById
DataQualityController	queryRuleListPaging
DataQualityController	queryRuleList
DataQualityController	queryExecuteResultListPaging
DataQualityController	getDatasourceOptionsById

Security center

Controller	Function	issue	status
AccessTokenController	queryProcessInstanceList	#10290	✅
AccessTokenController	generateToken	#10459
AccessTokenController	queryAccessTokenList	#10459
AccessTokenController	queryAccessTokenByUser	#10459
AccessTokenController	delAccessTokenById	#10459

TenantController	createTenant	#10467
TenantController	queryTenantlistPaging	#10467
TenantController	queryTenantlist	#10467
TenantController	updateTenant	#10467
TenantController	deleteTenantById	#10467
TenantController	verifyTenantCode	#10467

UsersController	createUser
UsersController	queryUserList
UsersController	updateUser
UsersController	delUserById
UsersController	grantProject
UsersController	grantProjectByCode
UsersController	revokeProject
UsersController	grantResource
UsersController	grantUDFFunc
UsersController	grantNamespace
UsersController	grantDataSource
UsersController	getUserInfo
UsersController	listUser
UsersController	listAll
UsersController	verifyUserName
UsersController	unauthorizedUser
UsersController	authorizedUser
UsersController	registerUser
UsersController	activateUser
UsersController	batchActivateUser

AlertGroupController	createAlertgroup
AlertGroupController	list
AlertGroupController	listPaging
AlertGroupController	queryAlertGroupById
AlertGroupController	updateAlertgroup
AlertGroupController	delAlertgroupById
AlertGroupController	verifyGroupName

AlertPluginInstanceController	createAlertPluginInstance
AlertPluginInstanceController	updateAlertPluginInstance
AlertPluginInstanceController	deleteAlertPluginInstance
AlertPluginInstanceController	getAlertPluginInstance
AlertPluginInstanceController	getAlertPluginInstance
AlertPluginInstanceController	verifyGroupName
AlertPluginInstanceController	listPaging

WorkerGroupController	saveWorkerGroup	#12072
WorkerGroupController	queryAllWorkerGroupsPaging	#12072
WorkerGroupController	queryAllWorkerGroups	#12072
WorkerGroupController	deleteWorkerGroupById	#12072
WorkerGroupController	queryWorkerAddressList	#12072

QueueController	queryList	#11067	✅
QueueController	queryQueueListPaging	#11067	✅
QueueController	createQueue	#11067	✅
QueueController	updateQueue	#11067	✅
QueueController	verifyQueue	#11067	✅

EnvironmentController	createEnvironment	#12075
EnvironmentController	updateEnvironment	#12075
EnvironmentController	queryEnvironmentByCode	#12075
EnvironmentController	queryEnvironmentListPaging	#12075
EnvironmentController	deleteEnvironment	#12075
EnvironmentController	queryAllEnvironmentList	#12075
EnvironmentController	verifyEnvironment	#12075

ClusterController	createProject
ClusterController	updateCluster
ClusterController	queryClusterByCode
ClusterController	queryClusterListPaging
ClusterController	deleteCluster
ClusterController	queryAllClusterList
ClusterController	verifyCluster

K8sNamespaceController	queryProjectListPaging
K8sNamespaceController	createNamespace
K8sNamespaceController	updateNamespace
K8sNamespaceController	verifyNamespace
K8sNamespaceController	delNamespaceById
K8sNamespaceController	queryUnauthorizedNamespace
K8sNamespaceController	queryAuthorizedNamespace
K8sNamespaceController	queryAvailableNamespaceList

Scheduler

Controller	Function	issue	status
SchedulerController	ALL	#11912

Are you willing to submit a PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct

The text was updated successfully, but these errors were encountered:

guodongym · 2022-06-12T12:55:02Z

Count me in @SbloodyS

SbloodyS · 2022-06-15T09:24:04Z

Some apis to be reconstructed have been added to this issue. Please reply the controller name and its function name you want to participate in and the issue you created. I will link it to this issue and assign it to you. Please feel free to take it. @liqingwang @guodongym @EricGao888

EricGao888 · 2022-06-15T09:29:46Z

@SbloodyS May I pick the monitor page? Thx #10458

SbloodyS · 2022-06-15T09:32:15Z

@SbloodyS May I pick the monitor page? Thx #10458

Sure. I've assigned it to you. Looking forward to your contribution.

qingwli · 2022-06-15T09:33:55Z

May I pick the Security center && login part? @SbloodyS

SbloodyS · 2022-06-15T09:35:50Z

AccessTokenController

What you want is the whole security center or just the AccessTokenController? Accesstokencontroller is only a part of the security center.

qingwli · 2022-06-15T09:39:21Z

Just AccessTokenController && LoginController first.

SbloodyS · 2022-06-15T09:41:24Z

Just AccessTokenController && LoginController first.

Sure. Please let me know after your issue is created.

SbloodyS · 2022-07-25T07:42:01Z

@SbloodyS hi, I got a rename problem. we want rename process to workflow. So which RequestMapping path is best choice?

v2/projects/{projectCode}/process-definition

projects/{projectCode}/workflow-definition

v2/projects/{projectCode}/workflow-definition

It's better to use v2/projects/{projectCode}/workflow-definition.

Zzih · 2022-08-31T07:27:00Z

I see that there are many functions inside the taskinstancecontroller，If I participate in the TaskInstanceController task, do I only need to modify the queryTaskListPaging forceTaskSuccess functions in TaskInstanceV2Controller?

Zzih · 2022-08-31T08:05:11Z

Plz assgin me Task Instance task.

SbloodyS · 2022-08-31T10:45:08Z

I see that there are many functions inside the taskinstancecontroller，If I participate in the TaskInstanceController task, do I only need to modify the queryTaskListPaging forceTaskSuccess functions in TaskInstanceV2Controller?

Yes. You are right.

SbloodyS · 2022-08-31T10:45:50Z

Plz assgin me Task Instance task.

I've assigned to you. Looking forward to your contribution.

Zzih · 2022-09-02T08:14:55Z

Plz assgin me Workflow Monitor task and help me review RP11747

lvshaokang · 2022-09-20T10:10:57Z

@zhongjiajie Would you assign me WorkerGroupController task, I will attempt to finish it, thk! BTW, should I create an issue first?

SbloodyS · 2022-09-21T01:22:56Z

Can I take K8sNamespaceController? or should i start contributing from issues with good first issue label?

It's recommanded to start from good first issue.

tracehh · 2022-09-21T01:52:38Z

@SbloodyS I take EnvironmentV2Controller #12075

zzzk1 · 2023-12-09T19:32:13Z

I would like to ask if there are any unfinished tasks for this issue that you can assign to me?

SbloodyS added help wanted Extra attention is needed feature new feature labels May 26, 2022

This comment was marked as outdated.

Sign in to view

SbloodyS changed the title ~~[API][Module Name] Suggest refactor the backend api~~ [API] Suggest refactor the backend api May 26, 2022

songjianet added this to the 4.0.0-alpha milestone May 26, 2022

SbloodyS changed the title ~~[API] Suggest refactor the backend api~~ [DSIP-7] [API] Suggest refactor the backend api May 26, 2022

SbloodyS added the DSIP label May 26, 2022

This comment was marked as outdated.

Sign in to view

apache deleted a comment from xushoubao May 27, 2022

This comment was marked as outdated.

Sign in to view

SbloodyS mentioned this issue May 30, 2022

[Doc] Fix api standard error #10292

Merged

SbloodyS added the backend label May 31, 2022

This comment was marked as outdated.

Sign in to view

EricGao888 mentioned this issue Jun 15, 2022

[Improvement][API] Refactor monitor page API #10458

Open

3 tasks

qingwli mentioned this issue Jun 15, 2022

[Improvement][API] Refactor AccessTokenController && LoginController API #10459

Open

3 tasks

EricGao888 mentioned this issue Jul 26, 2022

[Document-7870][docs] Modify the document system screenshot #10882

Closed

MichaelDeSteven mentioned this issue Jul 27, 2022

[Feature-11129] [API] Refactor org.apache.dolphinscheduler.api.controller.ProcessDefinitionController #11180

Closed

SbloodyS mentioned this issue Jul 30, 2022

[Improvement][swagger] Now the interface document returns only the generic template #11208

Closed

3 tasks

fuchanghai mentioned this issue Jul 30, 2022

[Improvement][all] Improvement checksyle #11211

Closed

3 tasks

SbloodyS mentioned this issue Aug 3, 2022

[Improvement][API] Change the way exceptions are handled in the API #11278

Closed

3 tasks

MichaelDeSteven mentioned this issue Aug 4, 2022

[API] Refactor org.apache.dolphinscheduler.api.controller.AlertPluginInstanceController #11298

Closed

3 tasks

ruanwenjun mentioned this issue Aug 8, 2022

[Feature-10983] [refactor] ResourcesController #10998

Closed

apache deleted a comment from xushoubao Aug 31, 2022

Zzih mentioned this issue Aug 31, 2022

[Feature][Api] Refactor org.apache.dolphinscheduler.api.controller.TaskInstanceController #11713

Closed

3 tasks

Zzih mentioned this issue Sep 2, 2022

[Feature][Api] Refactor org.apache.dolphinscheduler.api.controller.DataAnalysisController #11754

Open

3 tasks

zhongjiajie mentioned this issue Sep 15, 2022

[Feature][API] New restful API for workflow and schedule #11912

Merged

lvshaokang mentioned this issue Sep 20, 2022

[WIP][Feature][Api] Refactor api.controller.WorkerGroupV2Controller #12072

Open

3 tasks

apache deleted a comment from xushoubao Sep 21, 2022

SbloodyS mentioned this issue Nov 25, 2022

[Feature][API] New restful API for workflowInstance #12990

Merged

EricGao888 unpinned this issue Feb 9, 2023

zhongjiajie mentioned this issue May 15, 2023

[TOC] DSIP Collection #14102

Open

71 tasks

This was referenced Dec 16, 2023

[Feature][Api] Refactor org.apache.dolphinscheduler.api.controller.K8sNamespaceController #15328

Closed

[Feature][Api] Refactor org.apache.dolphinscheduler.api.controller.K8… #15329

Closed

zzzk1 mentioned this issue Dec 17, 2023

[Improvement][API] Refactor Task Group API #15331

Closed

3 tasks

apache deleted a comment from xushoubao Dec 19, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DSIP-7] [API] Suggest refactor the backend api #10257

[DSIP-7] [API] Suggest refactor the backend api #10257

SbloodyS commented May 26, 2022 •

edited

Loading

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

guodongym commented Jun 12, 2022

This comment was marked as outdated.

This comment was marked as outdated.

SbloodyS commented Jun 15, 2022

EricGao888 commented Jun 15, 2022 •

edited

Loading

SbloodyS commented Jun 15, 2022

qingwli commented Jun 15, 2022 •

edited

Loading

SbloodyS commented Jun 15, 2022

qingwli commented Jun 15, 2022

SbloodyS commented Jun 15, 2022

SbloodyS commented Jul 25, 2022

Zzih commented Aug 31, 2022

Zzih commented Aug 31, 2022

SbloodyS commented Aug 31, 2022

SbloodyS commented Aug 31, 2022

Zzih commented Sep 2, 2022 •

edited

Loading

lvshaokang commented Sep 20, 2022

SbloodyS commented Sep 21, 2022

tracehh commented Sep 21, 2022

zzzk1 commented Dec 9, 2023

[DSIP-7] [API] Suggest refactor the backend api #10257

[DSIP-7] [API] Suggest refactor the backend api #10257

Comments

SbloodyS commented May 26, 2022 • edited Loading

Search before asking

Description

Current API

Login page

Project manage page

Project

Workflow Monitor

Workflow Relationship

Workflow Definition

Workflow Instance

Task Definition

Task Instance

Task Group

Mornitor center

Datasource center

Resource center

Dataquality center

Security center

Scheduler

Are you willing to submit a PR?

Code of Conduct

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

guodongym commented Jun 12, 2022

This comment was marked as outdated.

This comment was marked as outdated.

SbloodyS commented Jun 15, 2022

EricGao888 commented Jun 15, 2022 • edited Loading

SbloodyS commented Jun 15, 2022

qingwli commented Jun 15, 2022 • edited Loading

SbloodyS commented Jun 15, 2022

qingwli commented Jun 15, 2022

SbloodyS commented Jun 15, 2022

SbloodyS commented Jul 25, 2022

Zzih commented Aug 31, 2022

Zzih commented Aug 31, 2022

SbloodyS commented Aug 31, 2022

SbloodyS commented Aug 31, 2022

Zzih commented Sep 2, 2022 • edited Loading

lvshaokang commented Sep 20, 2022

SbloodyS commented Sep 21, 2022

tracehh commented Sep 21, 2022

zzzk1 commented Dec 9, 2023

SbloodyS commented May 26, 2022 •

edited

Loading

EricGao888 commented Jun 15, 2022 •

edited

Loading

qingwli commented Jun 15, 2022 •

edited

Loading

Zzih commented Sep 2, 2022 •

edited

Loading