Merge pull request #141 from arayabrain/feature/update-developer-guid…

…eline update developer guidelines
arayabrain · Nov 20, 2023 · 25554c6 · 25554c6
2 parents ee76e4a + 07fcd38
commit 25554c6
Show file tree

Hide file tree

Showing 20 changed files with 147 additions and 365 deletions.
diff --git a/.vscode/extensions.json b/.vscode/extensions.json
@@ -2,6 +2,8 @@
   "recommendations": [
     "ms-python.black-formatter",
     "ms-python.flake8",
-    "ms-python.isort"
+    "ms-python.isort",
+    "dbaeumer.vscode-eslint",
+    "esbenp.prettier-vscode"
   ]
 }
diff --git a/docs/for_developers/architecture.md b/docs/for_developers/architecture.md
@@ -0,0 +1,61 @@
+## Architecture
+### Frontend
+- language: TypeScript
+- environment: Node.js (v20)
+- framework: React (v18)
+- middleware: Redux, Redux Toolkit, React Router
+
+#### Directory structure
+Following is the summary of directory structure.
+
+- frontend
+  - src
+    - api: handle API requests.
+    - components: React components.
+    - const: constants.
+    - pages: Components set as `element` in `Route` component of react-router.
+    - store: Redux store.
+      - slice
+        - Some
+          - SomeActions.ts: create actions like async thunk from api response.
+          - SomeSelectors.ts: select state from store.
+          - SomeSlice.ts: define reducers.
+          - SomeType.ts: define data type.
+          - SomeUtils.ts: define utility functions if needed.
+    - style: extra css files.
+    - utils: common utility functions.
+
+- api, components, store/slice directories are separated by its domain.
+
+### Backend
+- language: Python (v3.8)
+- framework: FastAPI
+- handle workflow by snakemake.
+
+#### Directory structure
+Following is the summary of directory structure.
+- studio
+  - alembic: files for database migration. database is used in multi-user mode.
+  - app
+    - `Snakefile`: File read by snakemake. It defines how to run workflow by language like Python.
+    - common: common modules(not for specific domain)
+      - core: define core functions.
+      - dataclass: define dataclasses that are passed between nodes or used as visualize outputs.
+      - db: database general configuration.
+      - models: database models.
+      - routers: define fastapi routers.
+      - schemas: define pydantic schemas for api requests and responses.
+      - wrappers: functions for algorithm nodes.
+    - optinist: modules specific to calcium imaging data processing domain. It has same structure as common.
+  - config: configuration files.
+    - `standalone-logging.yaml`: logging config for standalone mode. In standalone mode, logs are sent to streams.
+    - `logging.yaml`: logging config for multi-user mode. In multi-user mode, logs are sent to file (in `logs` dir). log file rotation settings are included.
+      - you can manage log configurations by editing these files.
+    - auth: put Firebase authentication secret files.
+    - docker: backend Dockerfiles for each environment.
+  - test_data: test data for unit tests.
+  - tests: unit tests.
+  - `__main__.py`: entry point of backend in production.
+  - `__main_unit__.py`: core main unit. called by `studio/__main__.py` and `main.py`.
+- `main.py`: entry point of backend in development.
+- `run_cluster.py`: entry point for CLI run.
diff --git a/docs/for_developers/index.rst b/docs/for_developers/index.rst
@@ -0,0 +1,16 @@
+.. _for_developers:
+
+***************
+FOR DEVELOPERS
+***************
+
+OptiNiSt welcomes your contributions.
+See the following guidelines before submitting Pull Requests.
+
+.. toctree::
+  :maxdepth: 2
+
+  style
+  pull_request
+  architecture
+  test
diff --git a/docs/for_developers/pull_request.md b/docs/for_developers/pull_request.md
@@ -0,0 +1,18 @@
+## Pull Request
+- GitHub Pull Request page
+  - [https://github.com/oist/optinist/pulls](https://github.com/oist/optinist/pulls)
+
+### Pre Commit
+- run following command before your first commit
+  ```
+  pre-commit install
+  ```
+  - Once installed, it automatically checks your coding style on every commits.
+
+### Branch rules
+- You can submit Pull Request by pushing new branch.
+  - Make sure the base branch is `develop-main`, and PR is to `develop-main`.
+  - You can't push to the `develop-main` branch directly, the branch is protected.
+  - Make sure new branch name is in following format (`xxx` is the name of the feature or bug you are working on.).
+    - `feature/xxx`
+    - `fix/xxx`
diff --git a/docs/utils/contributing.md → docs/for_developers/style.md b/docs/utils/contributing.md → docs/for_developers/style.md
@@ -1,16 +1,4 @@
-(Contributing)=
-# Contributing to OptiNiSt
-OptiNiSt welcomes your contributions.
-See the following guidelines before submitting Pull Requests.
-
-## Coding Style
-### Pre Commit
-- run following command before your first commit
-  ```
-  pre-commit install
-  ```
-  - Once installed, it automatically checks your code style on every commits.
-
+## Style Guides
 ### Python
 - Format all files using [black](https://black.readthedocs.io/en/stable/#).
 - Check the code for problems using [flake8](https://pypi.org/project/flake8/).
@@ -19,7 +7,14 @@ See the following guidelines before submitting Pull Requests.
 - Sort your import using [isort](https://github.com/PyCQA/isort).
 - These guides are checked by pre-commits, github workflow on submitting Pull Requests.
 
-#### VSCode Settings
+### TypeScript
+- Format all files using [prettier](https://prettier.io/).
+  - Rules are defined in `frontend/.prettierrc.json`.
+- Check the code for problems using [eslint](https://eslint.org/).
+  - Rules are defined in `frontend/.eslintrc.json`.
+
+### Editor support
+#### VSCode
 - If you are using VSCode, you can use features (like format/lint on save) by installing and activating the extensions in `.vscode/extensions.json`.
 - For these extensions, we provide example settings in `.vscode/settings.example.json`
   - To use this,

diff --git a/docs/for_developers/test.md b/docs/for_developers/test.md
@@ -0,0 +1,16 @@
+## Test
+- We have unit tests for both frontend and backend. They are automatically run by github workflow on submitting Pull Requests.
+- You can also run them locally.
+  - Frontend and Backend (Requires docker)
+    ```
+    make test_run
+    ```
+  - Frontend only
+    ```
+    yarn test
+    # or yarn test:ci
+    ```
+  - Backend only
+    ```
+    pytest
+    ```
diff --git a/docs/gui/visualize.md b/docs/gui/visualize.md
@@ -1,3 +1,4 @@
+(Visualize)=
 Visualize
 =================
 

diff --git a/docs/gui/workflow.md b/docs/gui/workflow.md
@@ -100,7 +100,7 @@ Select algorithms or analysis methods from the treeview on the left by clicking
 The left side of the window displays all available analysis methods. Clicking on the + mark adds the analysis nodes to the Workflow field. ROI detection tools (currently Suite2P, CaImAn and LCCD) are in the "Algorithm" category, and all other pre-installed analyses are in the "optinist" category.
 
 Let's start with sample TIFF data (`mouse2p_2_long.tiff`) and try Suite2P ROI detection.
-First, you need to determine the image you will use. Select your image as explained [above](#assigning-input-data-path).
+First, you need to determine the image you will use. Select your image as explained [above](#setting-input-data).
 Once it is selected, the name of the files is shown in the Image node.
 
 ### Parameter button and output button on the node
@@ -115,7 +115,7 @@ Each node has PARAM button and OUTPUT button.
 
 - **Editing Parameters:** Click on the PARAM button to view the parameters. You can edit them as needed. The names, types, and default values of the parameters are the same as the original algorithms. Refer to the original documentation to confirm the meaning of the parameters. The link list is available at [Implemented Analysis](https://optinist.readthedocs.io/en/latest/utils/implemented_analysis.html).
 
-- **Checking Results:** The OUTPUT button is for a quick check of the results. The button becomes active after the successful execution of the pipeline. For details about the charts, see [Inspecting the Images and the Plots on Visualize](#inspecting-the-images-and-the-plots-on-visualize).
+- **Checking Results:** The OUTPUT button is for a quick check of the results. The button becomes active after the successful execution of the pipeline. For details about the charts, see [](Visualize).
 
 ### Connecting Nodes
 

diff --git a/docs/host_for_multiuser/setup.md b/docs/host_for_multiuser/setup.md
@@ -98,8 +98,8 @@ Follow the steps below to setup `multiuser` mode.
     - `MYSQL_ROOT_PASSWORD` can be left commented.
 
 ### Setup Frontend
-1. Install node.js(version 18)
-    - https://nodejs.org/ja
+1. Install node.js(version 20)
+    - https://nodejs.org
 2. Install yarn
     ```bash
     npm install -g yarn

diff --git a/docs/index.rst b/docs/index.rst
@@ -36,3 +36,4 @@ Main Features
   gui/index
   host_for_multiuser/index
   utils/index
+  for_developers/index
diff --git a/docs/installation/create_virtualenv.md b/docs/installation/create_virtualenv.md
diff --git a/docs/installation/docker_for_developer.md b/docs/installation/docker_for_developer.md
@@ -32,12 +32,6 @@ docker compose -f docker-compose.dev.yml up
 
 - add `-d` option to make container run in background
 
-<!--
-## 2. Create virtualenv
-
-Under maintenance...
--->
-
 ## 2. Access to backend
 
 - Launch browser, and go to http://localhost:3000
@@ -51,4 +45,4 @@ Under maintenance...
 
 Done!
 
-If you will make PRs, please see the [](Contributing) section.
+If you will make PRs, please see the [](for_developers) section.
diff --git a/docs/installation/each_platforms_for_developer.md b/docs/installation/each_platforms_for_developer.md
@@ -28,9 +28,8 @@ Please follow instructions below.
 
 #### Install Node.js
 
-Get node with version 16 or 18
-- [Node.js Official](https://nodejs.org/ja)
-- version 20 is not supported yet.
+Get node with version 20
+- [Node.js Official](https://nodejs.org)
 
 You can also install node via [nvm](https://github.com/nvm-sh/nvm)
 
@@ -63,7 +62,7 @@ conda config --set channel_priority strict
 poetry install --no-root --with dev
 ```
 
-If you will make PRs, please see the [](Contributing) section.
+If you will make PRs, please see the [](for_developers) section.
 
 ### Set saving directory
 
@@ -72,12 +71,6 @@ Optinist default saving directory is `/tmp/studio`. If you reboot your PC, this
 export OPTINIST_DIR="your_saving_dir"
 ```
 
-<!--
-### 2. Create virtualenv
-
-Under maintenance...
--->
-
 ## 2. Run backend
 
 ```