[DSIP-2][Doc] Refactor the document

[zhongjiajie]

Description

For now, we documentation is not easy enough to read and maintain. I think documentation is also an important for OSS, it would be

General

• Sperate user_muanl/user_manal to multiple file with different level. Simply refactor document for DolphinScheduler 2.0 dolphinscheduler-website#453
• Add directory for our image file, start this task until all we markdown document is done.
• Find a way to use codebase repo directory icon files instead of copy-paste

Content

DRY

• remove duplicate content in the document, for example in installation we have duplicate download mysql dependent
• Many parameters of task have same definition, find a way to handle it. maybe we could see what github/doc do, https://github.com/github/docs

Missing document

• Chinese translation for ambari integration. @lukairui Chinese translation for ambari integration.  dolphinscheduler-website#494 Our community going to drop support of Apache Ambari in the further, so we do not need this anymore.
• How to contribute to our document, including dev and older doc.
• [Feature][doc] Add DSIP to documentation and issue template #6611
• 🔥 Missing Task: we missing some of our tasks in doc
  • SQOOP
  • condition. Simply refactor document for DolphinScheduler 2.0 dolphinscheduler-website#453
  • switch. [Feature][Doc] Add Task type switch #6561
  • waterdrop
  • pigeon. Simply refactor document for DolphinScheduler 2.0 dolphinscheduler-website#453
  • Task conditions screenshot add condition task img dolphinscheduler-website#461
• 🔥 Missing alert plugin
  • Alert plugin dingtalk add DingTalk alert plugin documentation dolphinscheduler-website#650
  • Alert plugin email
  • Alert plugin feishu
  • Alert plugin http
  • Alert plugin script
  • Alert plugin slack
  • Alert plugin wechat
• 🔥 Missing datasource plugin
  • Alert plugin clickhouse
  • Alert plugin db2
  • Alert plugin oracle
  • Alert plugin sqlserver
• Inside DolphinScheduler
  • How master node run
  • How worker node run
  • How master and worker community, and task lifecycle
• DolphinScheduler plugins development
• DolphinScheduler API
  • API docs and example, provider curl command line would be better.

To be classify

• user_doc/globalParam Move dev doc from docs to development dolphinscheduler-website#472
• All the spi document for develpment. [Feature][Doc] Move SPI documentation from docs to develpment #6577
• Unify dev-run and dev-quick-start Move dev doc from docs to development dolphinscheduler-website#472

To be update

• For now, task name much unify in the project
• introduction.md
• 🔥 Installation Refactor installation and add alert sub menu dolphinscheduler-website#465
• 🔥 Task Usage: we should add basic usage, example, notict about tasks
  • [Feature][Document] Add example and notice about task type Shell #8016
  • [Feature][Document] Add example and notice about task type SubProcess #8017
  • [Feature][Document] Add example and notice about task type Dependent #8018
  • [Feature][Document] Add example and notice about task type Stored Procedure #8019
  • [Feature][Document] Add example and notice about task type SQL #8020
  • [Feature][Document] Add example and notice about task type Spark #8021
  • [Feature][Document] Add example and notice about task type MapReduce #8022
  • [Feature][Document] Add example and notice about task type Python #8023
  • [Feature][Document] Add example and notice about task type Flink #8024
  • [Feature][Document] Add example and notice about task type HTTP #8025
  • [Feature][Document] Add example and notice about task type DataX #8026
  • [Feature][Document] Add example and notice about task type Pigeon #8027
  • [Feature][Document] Add example and notice about task type Conditions #8028
  • [Feature][Document] Add example and notice about task type Switch #8029
  • [Feature][Document] Add example and notice about task type SQOOP #8030
  • [Feature][Document] Add example and notice about task type seatunnel #8031
• Picture in English doc have Chinese content
  • design，[Bug] [Doc] Picture in designe EN doc have ZH content #7496
• architecture-design
  • Task queue. questions on architecture-design.md of 1.3.9 #7835
  • Architecture diagram. it seems that architecture diagram in the official website didn't update correspondingly #7306
• [Document] Upgrade 2.0 and dev branch screenshot to laster #7870

Syntax

• Use markdown table syntax instead of html table syntax <table>
• Find a better way to show our picture, for now it use html syntax in markdown file. I find our it can be show use markdown image syntax, such as this line

CI

• Add markdown link for our document
• Add chinese/english spelling ci for our doc
• Add document preview generation for each PR, example Add testing for docs in CI sqlfluff/sqlfluff#808 (comment) . I think it really important to reduce doc build error, such as this PR Correct dev leading page url wrong dolphinscheduler-website#454, we get error dev leading page url make all our document error.
• Add doc style check for our documentation, inspired by 1.3.9 doc-Add stand-alone component description dolphinscheduler-website#463 (comment). maybe we could use https://github.com/tidb-incubator/zh.md

[github-actions]

Hi:

• Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
• In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
• If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/zh-cn/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.

[zhongjiajie]

Hi @lukairui, I assigned ambari integration to your now.

[lukairui]

Hi @lukairui, I assigned ambari integration to your now.

Done，please review my pr.

[davidzollo]

Do you have any suggestions about Docs? please leave message

[davidzollo]

I suggest to add doc about how to use open API
it will be better if there are many parameters examples about each API interface

[zhongjiajie]

I suggest to add doc about how to use open API it will be better if there are many parameters examples about each API interface

Yeah, I think so. but I think we should find some way to auto-generate from our codebase instead of directly write doc by hand

[linjicong]

I noticed that some pictures are blurred

[zhongjiajie]

I noticed that some pictures are blurred

@linjicong Do you interested in this issue? I could create a new issue and assign to you if you like to do that.

[JinsYin]

We should also improve the document comments in the code, then use it in the website.

• https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/#common-setup-options
• https://spark.apache.org/docs/latest/configuration.html#application-properties

[zhongjiajie]

We should also improve the document comments in the code, then use it in the website.

• https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/#common-setup-options
• https://spark.apache.org/docs/latest/configuration.html#application-properties

You’re right do you have any idea to do it? Or you want to submit some PR to fix this?

[zhongjiajie]

The reason why we think the document is important, but why our community seems do not pay more attention on it

• Fix format and typo in 1.3.9 Doc dolphinscheduler-website#618 (comment)
• Fix format and typo in 1.3.9 Doc dolphinscheduler-website#618 (comment)

[Lucaszlei]

@zhongjiajie I am interested and able to help with the construction and improvement of the document. Next, I will gradually complete some of the problems mentioned above.

[zhongjiajie]

Hi, @sunnyz-l Thanks for play attention to the DolphinScheduler documents, looking forward to your contributions and you could ping me if you have any problems.

[linjicong]

@zhongjiajie
Is the file About_DolphinScheduler/About_DolphinScheduler.md standalone non-version module? This page now display 404 error when change language.

[zhongjiajie]

@zhongjiajie
Is the file About_DolphinScheduler/About_DolphinScheduler.md standalone non-version module? This page now display 404 error when change language.

Yeah, I check and you're right, it seems we have error in our introduction section. Do you want to correct it @linjicong

[linjicong]

@zhongjiajie
Is the file About_DolphinScheduler/About_DolphinScheduler.md standalone non-version module? This page now display 404 error when change language.

Yeah, I check and you're right, it seems we have error in our introduction section. Do you want to correct it @linjicong

I just submit a PR for it.

[zhongjiajie]

I add some independent new tasks for our tasks type usage. in #6535 (comment). You could search with the keyword "Task Usage" If anyone who interested in it, just feel free to take it and ping me in this issue.

[zhongjiajie]

I just submit a PR for it.

Thanks and I already review and left some commet in your PR @linjicong

[jj551]

@zhongjiajie architecture ! hi brother

[zhongjiajie]

architecture ! hi brother

@Kodalien I found out architecture has two children issue. which do you like to do?

1. Task queue. questions on architecture-design.md of 1.3.9 questions on architecture-design.md of 1.3.9 #7835
2. Architecture diagram. it seems that architecture diagram in the official website didn't update correspondingly it seems that architecture diagram in the official website didn't update correspondingly #7306

[jj551]

Hi,brother@zhongjiajie have alreadly done,but I think new architecture needs to develop to demo in modification

[zhongjiajie]

Hi,brother@zhongjiajie have alreadly done,but I think new architecture needs to develop to demo in modification

Hi @Kodalien which part should add, could you tell more detail about it.

[jj551]

need 2.0 architecture diagram demo @zhongjiajie

[zhongjiajie]

need 2.0 architecture diagram demo @zhongjiajie

I got your point, @caishunfeng do you have time to update our architecture diagram?

[caishunfeng]

need 2.0 architecture diagram demo @zhongjiajie

I got your point, @caishunfeng do you have time to update our architecture diagram?

The architecture diagram in the document still applies to version 2.0.

[zhongjiajie]

need 2.0 architecture diagram demo @zhongjiajie

I got your point, @caishunfeng do you have time to update our architecture diagram?

The architecture diagram in the document still applies to version 2.0.

Thanks @caishunfeng for clarifications, cc @Kodalien it seem we could keep it in, WDYT

[QuakeWang]

Hi, I will do this task, please assign to me : )

[zhongjiajie]

Hi, I will do this task, please assign to me : )

Hi @QuakeWang, thanks for you attend to this issue. but it is a collection of some known document tasks, you should point out which sub issue in #6535 (comment) you are interesting in instead of whole issue

[QuakeWang]

Hi, I will do this task, please assign to me : )

Hi @QuakeWang, thanks for you attend to this issue. but it is a collection of some known document tasks, you should point out which sub issue in #6535 (comment) you are interesting in instead of whole issue

OK, I will leave a comment under the specific issue.

[EricGao888]

We need to consistently improve the quality of our docs, especially English docs, see: #11385