docs: 更新文档 #1

README.en.md

@@ -0,0 +1,82 @@
+![LOGO](docs/resource/img/bk-job.png)
+---
+[![license](https://img.shields.io/badge/license-mit-brightgreen.svg?style=flat)](https://github.com/Tencent/bk-job/blob/master/LICENSE.txt) [![Release Version](https://img.shields.io/github/v/release/Tencent/bk-job?include_prereleases)](https://github.com/Tencent/bk-job/releases)
+
+English | [简体中文](README.md)
+
+> **Notice**: During the process of development, the 'master' branch might be in an *unstable state or unavailable state*.
+Please access the stable binary file via [releases](https://github.com/tencent/bk-job/releases) instead of 'master'.
+
+The BK-JOB is a ops script management and execution system with the capability of dealing with multiple tasks simultaneously. In addition to script execution, file distribution, cron jobs, and other basic operation environments, it is also capable of putting together all the individual tasks into an automated workflow. All tasks, as individual nodes, can be offered to upper-level or peripheral systems/platforms, making it an automated cross-system dispatching system.
+
+## Benefits
+
+### Dependable Dangerous Command Detection Ability
+
+As an underlying atomic operation platform made for server OS, the verification of user command's validity and safety is of critical importance! The platform allows for regular expression detection rules for dangerous commands in various script languages, and provides a log for intercepted operations. Even when a command is made by a peripheral system via API, it'll be intercepted immediately, making the server more secure!
+
+### Comprehensive Script Version Management
+
+Cloud management of script versions, which is accordant with the modern concept of collaboration. Aided by the convenient platform and version management feature, BK-JOB users can share script resources, control the releasing status of various versions, and reduce loss if there's a security vulnerability.
+
+### Arrange Jobs for All Scenarios
+
+When an operation scenario requires multiple interconnected steps, clicking them one by one is too inefficient! Besides, it is inconvenient for later usage and maintenance.
+
+The task management feature offers an optimal solution. Users can set up the necessary steps on the "Job Template" before making up an "Executable Plan" as required by the scenario; As such, job templates and its instances are set apart rather than welded together, which is also beneficial to later management and maintenance.
+
+### The Authentic Cron Jobs
+
+Linux's original crontab job feature is preserved, allowing the maintenance team to access them easily. The friendly monitoring feature allows you to easily control and assess the situation of scheduled tasks.
+
+### Scalable File Source Management
+
+When it comes to file distribution, we not only use remote server and local files as source of transmission, we also need to retrieve files from different file systems/services such as object storage, FTP, and Samba.
+
+To meet the diversified demands, we have enabled the file source plug-in feature, which allows developers to develop plugins for the file management modules on the platform according to their own file system. As such, they can distribute files from various file systems.
+
+
+BK-JOB provides quick execution, task arrangement, cron job, and a wide range of core services, meeting the demands of all environments:
+- **Quick Execution**: Offers a temporary, versatile, quick access.
+- **Task Arrangement**: Transforms repetitive operations into "jobs" using the arrangement feature to make them easy to use and manage.
+- **Scheduled Execution**: Allows users to make up periodic or one-off execution plans according to the logical needs of their business.
+- **Script Management**: Manage scripts in a cloud-based mode, which further facilitates job arrangement and the flexibility of peripheral system scheduling.
+- **Account Management**: The execution account of that manages server OS, such as Linux root or Windows administrator, etc.
+- **Notification**: Allows businesses to send notifications on the execution result for various task statuses.
+- **File Source Management**: Enables file source plugins, allowing it to retrieve and transfer files from various file systems.
+- **Operation Analysis**: Displays platform operation statistics, which allows the administrator to have an all-around insight into the operation of the platform.
+- **Platform Management**: A versatile platform management tool whose features include, but not limited to, information modification, message channel settings, rules for detecting dangerous commands, feature limitation settings, public script management, background service status display, etc.
+
+## Overview
+
+- [Architecture Design](docs/overview/architecture.en.md)
+- [Code Directory](docs/overview/code_framework.en.md)
+- [Design Philosophy](docs/overview/design.en.md)
+
+## Features
+
+For more information, please check Blueking's official website [Platform Product Overview](https://bk.tencent.com/docs/document/6.0/125/5748)
+
+## Getting started
+- [Download and Compile](docs/overview/source_compile.en.md)
+- [Installation Setup](docs/overview/installation.en.md)
+
+## Support
+1. [GitHub Community](https://github.com/Tencent/bk-job/discussions)
+
+## BlueKing Community
+- [BK-BCS](https://github.com/Tencent/bk-bcs): Blueking Container Service is a container-based basic service platform that provides management service to microservice businesses.
+- [BK-BCS-SaaS](https://github.com/Tencent/bk-bcs-saas): Blueking Container Platform SaaS is based on two modes: original Kubernetes and self-developed Mesos, offering users highly scalable, flexible container service.
+- [BK-CI](https://github.com/Tencent/bk-ci): Blueking Continuous Integration platform is a free, open source CI service, which allows developers to automatically create - test - release workflow, and continuously, efficiently deliver their high-quality products.
+- [BK-CMDB](https://github.com/Tencent/bk-cmdb): BlueKing CMDB is an enterprise-level management platform designed for assets and applications.
+- [BK-PaaS](https://github.com/Tencent/bk-PaaS): Blueking PaaS is an open development platform that allows developers to efficiently create, develop, set up, and manage SaaS apps.
+- [BK-SOPS](https://github.com/Tencent/bk-sops): Blueking SOPS is a system that features workflow arrangement and execution using a graphical interface. It's a lightweight task scheduling and arrangement SaaS product of the Blueking system.
+
+## Contributing
+- For more information about bk-job's fork management, issue and PR rules, please check [Contributing](CONTRIBUTING.md).
+- [Tencent Open Source Incentive Plan](https://opensource.tencent.com/contribution) encourages developers to engage and contribute in the community. Join us now!
+
+
+## License
+BK-JOB is based on the MIT agreement. For more information, please check [LICENSE](LICENSE.txt).
+

README.md

@@ -2,6 +2,7 @@
 ---
 [![license](https://img.shields.io/badge/license-mit-brightgreen.svg?style=flat)](https://github.com/Tencent/bk-job/blob/master/LICENSE.txt) [![Release Version](https://img.shields.io/github/v/release/Tencent/bk-job?include_prereleases)](https://github.com/Tencent/bk-job/releases)
 
+[English](README.en.md) | 简体中文
 
 > **重要提示**: `master` 分支在开发过程中可能处于 *不稳定或者不可用状态* 。
 请通过[releases](https://github.com/tencent/bk-job/releases) 而非 `master` 去获取稳定的二进制文件。

UPGRADE.en.md

@@ -0,0 +1,28 @@
+## About Updates
+
+English | [简体中文](UPGRADE.md)
+
+### Version 3.3.4.x
+This version is joined by a new update tool (upgrader-3.3.4.x.jar, in the same directory with other binary packages). It needs to be run before and after restarting the process of Job binary Jar file update (all microservice processes), which is twice in total for the tool, in order to transfer permission data and encrypted account data.
+
+**Compatibility**  
+It is used for updates from 3.x.x.x to 3.3.4.x or higher. It doesn't need to be executed when updating within the 3.3.4.x version (e.g. 3.3.4.0-->3.3.4.3)
+
+**Using the Tool**   
+Launch the tool with the command: `java -Dfile.encoding=utf8 -Djob.log.dir=path/to/log/dir -Dconfig.file=/path/to/config/file -jar upgrader-[x.x.x.x].jar [fromVersion] [toVersion] [executeTime]` Next, enter the specific parameters of the version update task based on the command line prompts, then run the update tool.  
+
+/path/to/log/dir The log saving path for specified tools, mostly ${BK_HOME}/logs/job  
+/path/to/config/file The configuration file loaded by specific tools. The configuration file is automatically generated by the setup script. Its path is: ${BK_HOME}/etc/job/upgrader/upgrader.properties
+fromVersion is the current platform version, e.g. 3.2.7.3  
+toVersion is target platform version, e.g. 3.3.4.0  
+executeTime is the execution time of the update task, its has BEFORE_UPDATE_JOB and AFTER_UPDATE_JOB values. If you're running this tool before the platform update, enter BEFORE_UPDATE_JOB. If its after the update, enter AFTER_UPDATE_JOB  
+
+Example:  
+Before Job process update:    
+```shell script
+/opt/java/bin/java -Dfile.encoding=utf8 -Djob.log.dir=/data/bkee/logs/job -Dconfig.file=/data/bkee/etc/job/upgrader/upgrader.properties -jar upgrader-3.3.4.0.jar 3.2.7.3 3.3.4.0 BEFORE_UPDATE_JOB  
+```
+After Job process update:  
+```shell script
+/opt/java/bin/java -Dfile.encoding=utf8 -Djob.log.dir=/data/bkee/logs/job -Dconfig.file=/data/bkee/etc/job/upgrader/upgrader.properties -jar upgrader-3.3.4.0.jar 3.2.7.3 3.3.4.0 AFTER_UPDATE_JOB
+```

UPGRADE.md

@@ -1,4 +1,7 @@
 ## 升级说明
+
+[English](UPGRADE.en.md) | 简体中文
+
 ### 版本3.3.4.x
 该版本增加一个升级工具（upgrader-3.3.4.x.jar，与其他二进制包在同一目录），需要在更新Job二进制Jar文件重启进程（所有微服务进程）前后用不同参数分别执行一次，工具总共执行两次，用于迁移权限数据和账号加密数据。
 

docs/install/backend.en.md

@@ -0,0 +1,62 @@
+# Backend Microservice Setup  
+
+English | [简体中文](backend.md)
+
+There are 10 microservices and 1 file source access point (job-file-worker) on bk-job's backend. The compiled products are listed below:
+
+|Package Name | Description |
+|:---- |:----|
+job-analysis-{version}.jar | Statistical Analysis Microservice Springboot.jar 
+job-backup-{version}.jar   | Backup Management Microservice Springboot.jar  
+job-config-{version}.jar   | Configuration Center Microservice Springboot.jar  
+job-crontab-{version}.jar  | Scheduled Task Microservice Springboot.jar  
+job-execute-{version}.jar  | Job Execution Microservice Springboot.jar  
+job-file-gateway-{version}.jar | File Gateway Microservice Springboot.jar  
+job-file-worker-{version}.jar | File Source Access Point Springboot.jar  
+job-gateway-{version}.jar | Backend Gateway Microservice Springboot.jar  
+job-logsvr-{version}.jar | Log Management Microservice Springboot.jar  
+job-manage-{version}.jar | Job Management Microservice Springboot.jar  
+job-ticket-{version}.jar | Ticket Management Microservice Springboot.jar  
+
+**Process**  
+1. Check System Dependencies
+Make sure that the basic Blueking environment has been properly configured, including PaaS, user management, ESB, CMDB, GSE, and IAM. Make sure the opened ESB interface and backend interface is working properly.
+
+2. Check Basic Environment
+Make sure that the bk-job's component dependencies (MySQL, Redis, etc.) have been set up in the previous step.
+
+3. Configure and Launch Service
+(1) Create a directory structure as shown below. Place the compiled products and the files under Scripts and Support-files directories in the corresponding locations.
+```
+|- /data/bkee/job        # Home Directory
+  |- support-files       # Resource Files
+  |- backend             # The program that stores backend microservices and file source access points
+    |- job-manage        # job-manage Microservice Jar package and ops script. There are 11 directories in total, which have already been introduced above.
+      |- job-manage.jar  # job-manage Microservice's SpringBoot.jar
+      |- bin  
+        |- job-manage.sh # job-manage Microservice's ops script
+    |- job-xxx
+```
+(2) After ensuring JAVA_HOME's environment variables have been properly configured, start all microservice processes in sequence:
+Example of starting microservice:
+```shell script
+cd /data/bkee/job/backend/job-config/bin
+./job-config.sh start
+```  
+
+Sequence:  
+job-config (The configuration center is essential for other microservices. Therefore, it needs to be started first.)  
+job-manage  
+job-backup  
+job-logsvr  
+job-ticket  
+job-file-gateway  
+job-execute  
+job-crontab  
+job-gateway  
+job-file-worker  
+job-analysis  
+The sequence mentioned above is the optimal one recommended as per invocation relations. If you need to start all services immediately, after launching job-config, you can start other microservices and job-file-worker at the same time. Once it's completed, check the log output and process state and restart the failed processes.  
+When the microservice is started, log directory will be automatically created:  
+${BK_HOME}/logs/${PROJECT_NAME}/${SERVICE_NAME}/
+In this example, BK_HOME is /data/bkee, PROJECT_NAME is job, SERVICE_NAME is the name of the microservice. Job-file-worker doesn't count as a microservice, but the way of launching it is the same with other microservices.  

docs/install/backend.md

@@ -1,4 +1,7 @@
 # 后端微服务部署  
+
+[English](backend.en.md) | 简体中文
+
 bk-job后端共有10个微服务和一个文件源接入点（job-file-worker），编译产物如下表所示：
 
 |包名称 | 描述 |

docs/overview/architecture.en.md

@@ -0,0 +1,36 @@
+# Blueking (BK-JOB) Architecture Design
+
+English | [简体中文](architecture.md)
+
+![Architecture](../resource/img/architecture.png)
+
+The BlueKing Platform (aka **bk-job** ) is written in multiple languages including java/js/html/python/shell/gradle, featuring separation of frontend and backend, with highly-accessible, scalabe, service-oriented architecture:
+
+- **Frontend (FrontEnd):**
+
+  Features ES6 grammar, MVVM architecture vue.js, and webpack4 package.
+
+- **Backend (BackEnd):** 
+
+Written in Java, featuring microservice architecture of Spring Cloud. Here is a brief rundown on each of the microservice modules:
+
+  - **job-config:** Configuration Center: Manages and coordinates the configuration information of all microservices.
+  - **job-gateway:** Microservice Gateway: In charge of authentication, traffic throttling, routing requests, etc.
+  - **job-manage:** Job Management Microservice: In charge of managing the resources on the platform, including script, account, job template, execution plans, notification, and general settings.
+  - **job-execute:** Job Execution Microservice: Serves Blueking GSE by submitting file distribution/script execution tasks, retrieving task logs, and converting task statuses.
+  - **job-logsvr:** Log Management Microservice: Connects to the underlying MongoDB. In charge of storing the logs generated from script execution, file distribution, and source file downloading.
+  - **job-crontab:** Scheduled Task Microservice. Task scheduling and management of BK-job based on the Quartz engine.
+  - **job-backup:** Backup Management Microservice: Imports and exports jobs in bk-job and archives job execution history on a regular basis.
+  - **job-analysis:** Statistical Analysis Microservice. Provides backend access to job error message on the main page and operation analysis module; dispatches timed tasks; acquires and analyzes the metadata from other microservices; generates analysis result and statistics; provides data for the operation of bk-job; improves accessibility of the platform.
+  - **job-ticket:** Credential Management Microservice: Provides a variety of credentials for the third party file source connected to the file gateway; stores, encrypts and decrypts credentials.
+  - **job-file-gateway:** File Gateway Microservice: Connects with various third party file sources via FileWorker (object-based storage, file system storage, etc.); schedules the file downloading tasks from file source; works with the execution engine to distribute files from third party file source.
+  - **job-file-worker:** An access point, an expandable module independent of other bk-job backend microservice; multiple instances can be set up; communicate with the file gateway to connect to various types of third party file sources; executes file downloading tasks.
+
+- **Resource Service (Resource):** Provides the necessary basic middleware for storage.
+    - **Consul:** Used as a service discovery server; You need to build a Consul Server and install Consul on the machine where bk-job microservices are deployed, and run it as Agent.
+    - **RabbitMQ:** Core Message Queue Service: bk-job's task execution engine uses RabbitMQ to deliver the message of task status changes.
+    - **MySQL:** bk-job's primary database. Uses mysql 5.7.2 to store the relational data of all microservices listed above.
+    - **Redis:** Core Service Cache 4.0.14: Provides host information for distributed locks and cache service.
+    - **MongoDB:**  BK-job's log database. MongoDB 4.2.2. Used to store the log data generated from script execution/file distribution.
+    - **NFS:** Used to store user-uploaded local files in local file distribution scenario and store temporary files generated from job import and export.
+

docs/overview/architecture.md

@@ -1,5 +1,7 @@
 # 蓝鲸作业平台(BK-JOB)架构设计
 
+[English](architecture.en.md) | 简体中文
+
 ![Architecture](../resource/img/architecture.png)
 
 蓝鲸作业平台(简称**bk-job** )是基于 java/js/html/python/shell/gradle等多种语言编写实现的，采用完全前后端分离，具备高可用可扩展的服务架构设计：