update all docs except transform

docs/getting-started/configuration.md

@@ -10,13 +10,13 @@ This document describes the configuration file parameters for MantisTable UI. Th
 
 1. **Copy the Example File**
 
-   First, make a copy of the `example.env` file. This can be done using the following command in a terminal:
+   First, make a copy of the `.env.example.prod` file. This can be done using the following command in a terminal:
 
    ```sh
-   cp example.env .env
+   cp .env.example.prod .env
    ```
 
-   The `example.env` is located inside the main folder:
+   The `.env.example.prod` is located inside the main folder:
 
    ```bash
     mantistable-ui
@@ -28,7 +28,8 @@ This document describes the configuration file parameters for MantisTable UI. Th
     ├── components.json
     ├── docker-compose.yml
     ├── drizzle.config.ts
-    ├── example.env
+    ├── .env.example.dev
+    ├── .env.example.prod
     .
    ```
 
@@ -37,56 +38,61 @@ This document describes the configuration file parameters for MantisTable UI. Th
    Open the newly created `.env` file in a text editor of your choice. Modify the parameters as needed to fit your environment. Below is an example of how the file might look after customization:
 
    ```env
-   DATABASE_URL="postgresql://postgres:yourpassword@yourhost:5432/yourdatabase"
-   POSTGRESQL_PASS="your_postgres_password"
-   POSTGRES_HOST="your_postgres_host"
-   POSTGRES_PORT="your_postgres_port"
-   POSTGRES_DB="your_postgres_db"
-   POSTGRES_USER="your_postgres_user"
-
-   MONGO_INITDB_ROOT_USERNAME="your_mongo_username"
-   MONGO_INITDB_ROOT_PASSWORD="your_mongo_password"
-   
-   STI_HOST="http://your.sti.host:port"
-   
-   PLUGINS_PORT="your_plugins_port"
+   # PostgreSQL
+   POSTGRESQL_PASS="..."
+   POSTGRES_HOST="..."
+   POSTGRES_PORT="..."
+   POSTGRES_DB="..."
+   POSTGRES_USER="..."
+
+   # MongoDB
+   MONGO_INITDB_ROOT_USERNAME="..."
+   MONGO_INITDB_ROOT_PASSWORD="..."
+
+   # STI
+   STI_HOST_UPLOAD_AND_PROCESS="..."
+   STI_HOST_DOWNLOAD_ANNOTATIONS="..."
+
+   # Plugins
+   PLUGINS_HOST="..."
+   NEXT_PUBLIC_PLUGINS_HOST="..."
+   PLUGINS_PORT="..."
+
+   # Mantis
+   MANTIS_OUTSIDEPORT="..."
+
+   # Mantistablex Variables
+   GPT_KEY="..."
+   GPT_ENDPOINT="..."
    ```
 
 ## Configuration Parameters
 
 ### PostgreSQL
 
-- **`POSTGRESQL_URL`**
-
-  ```plaintext
-  DATABASE_URL="postgresql://postgres:password@localhost:port/mantistableui"
-  ```
-
-  Specifies the URL for connecting to the PostgreSQL database.
-
 - **`POSTGRESQL_PASS`**
 
   ```plaintext
   POSTGRESQL_PASS="password!"
   ```
 
-  The password for the PostgreSQL database user. This should match the password specified in the `DATABASE_URL`.
+  The password for the PostgreSQL database.
 
 - **`POSTGRES_HOST`**
 
   ```plaintext
   POSTGRES_HOST="localhost"
   ```
 
-  The hostname or IP address of the PostgreSQL server.
+  The hostname or IP address of the PostgreSQL server. The service is available also inside the Docker Network, so it's possible to specify `postgres`, which is the name of the service.
 
-- **`POSTGRES_POR`**
+- **`POSTGRES_PORT`**
 
   ```plaintext
   POSTGRES_PORT="5432"
   ```
 
-  The port on which the PostgreSQL server is running.
+  The port on which the PostgreSQL server is running. `5432` is the default one.
 
 - **`POSTGRES_DB`**
 
@@ -124,48 +130,113 @@ This document describes the configuration file parameters for MantisTable UI. Th
 
 ### External STI Services
 
-- **`STI_HOST`**
+- **`STI_HOST_UPLOAD_AND_PROCESS`**
+
+  ```plaintext
+  STI_HOST_UPLOAD_AND_PROCESS="http://sti_upload_and_process:5000"
+  ```
+
+  The address of the STI endpoint available to upload and process a single table. Replace this with the appropriate URL for your setup.
+
+  - **`STI_HOST_DOWNLOAD_ANNOTATIONS`**
 
   ```plaintext
-  STI_HOST="http://local:5042"
+  STI_HOST_DOWNLOAD_ANNOTATIONS="http://sti_download:5000"
   ```
 
-  Specifies the hostname, URL or IP address for an STI service. Replace this with the appropriate URL for your setup.
+  The address of the STI endpoint available to download the annotated table. Replace this with the appropriate URL for your setup.
 
   :::warning
   The external Semantic Table Interpretation service must expose APIs as indicated in [External STI Approach](/docs/sti/external-sti-approach) page.
   :::
 
 ### Plugins
 
+- **`PLUGINS_HOST`**
+
+  ```plaintext
+  PLUGINS_HOST="http://api:5000"
+  ```
+
+  The address of the plugin service. The address specified here is internal in the docker network, such as `http://api:5000`, which specify the container name.
+
+- **`NEXT_PUBLIC_PLUGINS_HOST`**
+
+  ```plaintext
+  NEXT_PUBLIC_PLUGINS_HOST="http://localhost:5001"
+  ```
+
+  The address of the plugin service. The address specified here is external to the docker network, such as `http://localhost:5001`, because it is used in the frontend module.
+
 - **`PLUGINS_PORT`**
 
   ```plaintext
-  PLUGINS_PORT="port"
+  PLUGINS_PORT="5001"
   ```
 
   The port on which the plugins service (Doker container) will run.
 
-## Configuration File Example
+### MantisTable
 
-Below is a complete example of a configuration file with all parameters:
+- **`MANTIS_OUTSIDEPORT`**
 
-```plaintext
-DATABASE_URL="postgresql://postgres:password!@localhost:5432/mantistableui"
-POSTGRESQL_PASS="password!"
-POSTGRES_HOST="localhost"
-POSTGRES_PORT="5432"
-POSTGRES_DB="mantistableui"
-POSTGRES_USER="postgres"
+  ```plaintext
+  MANTIS_OUTSIDEPORT="3000"
+  ```
+
+  The port on which the frontend platform service (Doker container) will run.
+
+### Mantistablex
+
+- **`GPT_KEY`**
+
+  ```plaintext
+  GPT_KEY="..."
+  ```
+
+  The private key from GPT APIs used to execute the mantistablex plugin.
+
+- **`GPT_ENDPOINT`**
+
+  ```plaintext
+  GPT_ENDPOINT="..."
+  ```
 
-MONGO_INITDB_ROOT_USERNAME="root"
-MONGO_INITDB_ROOT_PASSWORD="password!"
+  The endpoint from GPT APIs used to execute the mantistablex plugin.
 
-STI_HOST="http://unimib.it:5042"
+## Configuration File Example
 
-PLUGINS_PORT="5001"
+Below is a complete example of a configuration file with all parameters:
+
+```plaintext
+   # PostgreSQL
+   POSTGRESQL_PASS="table_ui2024!"
+   POSTGRES_HOST="postgres"
+   POSTGRES_PORT="5432"
+   POSTGRES_DB="postgrestableui"
+   POSTGRES_USER="postgres"
+
+   # MongoDB
+   MONGO_INITDB_ROOT_USERNAME="root"
+   MONGO_INITDB_ROOT_PASSWORD="export2024!"
+
+   # STI
+   STI_HOST_UPLOAD_AND_PROCESS="https://selbat.datai.disco.unimib.it/dataset/createWithArray?token=selBat_demo_2023"
+   STI_HOST_DOWNLOAD_ANNOTATIONS="http://vm.chronos.disco.unimib.it/dataset/${datasetName}/table/${table_id}?token=alligator_demo_2023"
+
+   # Plugins
+   PLUGINS_HOST="http://api:5000"
+   NEXT_PUBLIC_PLUGINS_HOST="http://localhost:5001"
+   PLUGINS_PORT="5001"
+
+   # Mantis
+   MANTIS_OUTSIDEPORT="3000"
+
+   # Mantistablex Variables
+   GPT_KEY="..."
+   GPT_ENDPOINT="..."
 ```
 
 ## Conclusion
 
-The configuration file for MantisTable UI is **crucial** for ensuring proper connections to databases and external services. By correctly setting these parameters, you ensure the smooth operation and integration of MantisTable UI within your environment.
+The configuration file for MantisTable UI is **crucial** for ensuring proper connections to databases and external services. By correctly setting these parameters, you ensure the smooth operation and integration of MantisTable UI within your environment.

docs/plug-in/add-on.md

@@ -3,3 +3,95 @@ sidebar_position: 2
 ---
 
 # Add-on
+
+Add-on plugins enable MantisTable UI to perform arbitrary tasks on the annotated table. An example of `add-on` plugin is the lexicalization of the annotated table (`mantistablex`).
+
+### General Guidelines
+
+1. **Isolation:** Each plugin should operate independently and not interfere with other plugins or the core functionality of MantisTable UI.
+2. **Configuration:** Plugins should allow configuration through environment variables or configuration files.
+3. **Documentation:** Provide clear documentation for each plugin, including usage instructions, dependencies, and configuration options.
+
+### Export Plugin Example
+
+**Structure:**
+
+```
+/addon-plugin-example
+  mantistablex.py
+  config.json
+  input.json
+  inputData.json
+  input.html
+  output.html
+  README.md
+```
+
+**export.py:**
+
+```python
+### Read Annotated Table
+table_input_path = f"{os.path.dirname(os.path.realpath(__file__))}/input.json"
+file_input = open(table_input_path, encoding="utf-8")
+original_table = json.load(file_input)
+# Read Input Data for the Plugin
+input_data_path = f"{os.path.dirname(os.path.realpath(__file__))}/inputData.json"
+file_input_data = open(input_data_path, encoding="utf-8")
+input_data: dict = json.load(file_input_data)
+
+# ADD-ON CODE
+
+# Write output on output.html
+output_file_path = f"{os.path.dirname(os.path.realpath(__file__))}/output.html"
+```
+
+**config.json:**
+
+```json
+{
+  "type": "addon",
+  "name": "mantistablex",
+  "description": "MantisTableX plugin converts structured data into natural language descriptions. Given an input table with annotations specifying the relationships and significance of its data, the tool generates coherent textual summaries or narratives that convey the table's information in readable and contextually appropriate sentences.",
+  "entry_file": "mantistablex.py",
+  "output": "output.html"
+}
+```
+
+The entry file is specified by the `entry_file` field in the `config.json` file and it represents the entry point of the plugin, because it is the file that is executed to apply the plugin. Once the plugin is selected, the annotated table (defined in the output of the STI sytem ~ [External STI Approach](/docs/sti/external-sti-approach)) is injected in the `input.json` file. In this case, there is also the possibility to define an html form to enable the user to send input data to the add-on plugin for its execution. To add input data, it's important to define the form in the `input.html` file using a `<div>` as root element. Then, the `entry_file` is applied using the `subprocess` module in python and, finally, the output is read in the file defined by the `config.json` using the `output` field (usually defined as `output.html`) in the root folder. For this reason it is very important to use the following structure to identify the correct file path in the plugin:
+
+```python
+table_input_path = f"{os.path.dirname(os.path.realpath(__file__))}/input.json" # input table
+input_data_path = f"{os.path.dirname(os.path.realpath(__file__))}/inputData.json" # input data
+output_file_path = f"{os.path.dirname(os.path.realpath(__file__))}/output.html" # output html
+```
+
+The plugin is executed on an isolated environment, so you can't install your own libraries but some are already installed:
+
+```
+tqdm
+rdflib
+pandas
+numpy
+requests
+psycopg2
+pymongo
+python-multipart
+```
+
+Add-on plugins are very powerful because the definition of the UI to display the data entry form and the plugin's output is left to the plugin developer, who will define the UI using HTML Markup language and inline CSS to style the content.
+
+## Using Plugins in MantisTable UI
+
+1. **Install the Plugin:**
+
+   - Place the plugin files in the appropriate directory as specified in the configuration file.
+   - Ensure all dependencies are installed and properly configured.
+
+2. **Configure the Plugin:**
+
+   - Update the `config.json` file with the necessary configuration parameters.
+
+3. **Access Plugin Functionality:**
+   - Export plugins can be accessed through the export menu in the UI.
+   - Add-on plugins are typically accessed through semantic processing options.
+   - Transformation plugins can be applied through the data transformation tools in the UI.