Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CU-86c029q15 - Update Windows Installation Scripts #312

Merged
merged 7 commits into from
Sep 11, 2024

Conversation

sushantpatil1214
Copy link
Collaborator

@sushantpatil1214 sushantpatil1214 commented Sep 3, 2024

Summary by CodeRabbit

  • New Features

    • Implemented new environment variables and configurations for improved service orchestration in Docker setups.
    • Enhanced directory creation and permission management for PostgreSQL and configuration directories.
    • Added multiple HTTP port configurations essential for service communication.
  • Bug Fixes

    • Updated file paths for configuration files in the deployment scripts to ensure correct application behavior.
  • Documentation

    • Enhanced clarity in environment configuration scripts with new variables and structured paths.
  • Chores

    • Streamlined deployment scripts by removing redundant operations and improving logging for better user experience.

@rcrichton
Copy link
Member

Copy link
Contributor

coderabbitai bot commented Sep 3, 2024

Caution

Review failed

The pull request is closed.

Walkthrough

The recent changes include updates to the .gitignore file, enhancements to environment variable configurations for Docker deployments, modifications to file path handling in the JeMPI API, and adjustments to PowerShell scripts for file management and directory creation. Additionally, new environment variables have been introduced for API configurations and database management, improving the organization and adaptability of the JeMPI framework.

Changes

File Path Change Summary
.gitignore Added entries to ignore specific configuration and CSV files related to Windows Docker WSL setup.
JeMPI_Apps/JeMPI_LibAPI/src/main/java/org/jembi/jempi/libapi/BackEnd.java Updated file path handling for CSV uploads and configuration retrieval based on environment variables.
devops/windows/base-docker-linux/conf/env/conf-env-high-1-pc.template Introduced new environment variables and modified existing ones for service orchestration and database management.
devops/windows/base-docker-linux/conf/env/conf-env-low-1-pc.template Added new environment variables for API configuration management.
devops/windows/base-docker-linux/conf/env/create-env-linux-high-1.sh Updated environment variables for multi-node deployment, including database and API configurations.
devops/windows/base-docker-linux/conf/env/create-env-linux-low-1.sh Enhanced project path and database configurations with new variables for clarity and organization.
devops/windows/base-docker-linux/helper/scripts/d-stack-01-create-dirs.sh Added directory creation for PostgreSQL and configuration files with appropriate permissions.
devops/windows/run--base-docker-wsl/start.ps1 Modified file copy commands to reference new configuration files and locations.
JeMPI_Apps/*/docker/Dockerfile Set default JAVA_VERSION to 21.0.1-tem for consistency across various Dockerfiles.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant API
    participant Config
    participant Database

    User->>API: Request data
    API->>Config: Fetch configuration
    Config-->>API: Return configuration
    API->>Database: Query data
    Database-->>API: Return data
    API-->>User: Send response
Loading

🐰 In the meadow where I hop and play,
New configs sprout like flowers in May.
With paths and ports all set just right,
My data dances in the bright sunlight.
Hooray for changes, let’s cheer and sing,
For every hop, a joyful spring! 🌼

Possibly related PRs


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 8

Outside diff range, codebase verification and nitpick comments (1)
devops/windows/base-docker-linux/conf/env/create-env-linux-low-1.sh (1)

42-47: Port Conflict Issue Detected

The configuration sets the same port number (50000) for multiple services across different scripts, which can lead to port conflicts. Each service should have a unique port number to avoid such conflicts.

  • Affected files include:
    • devops/windows/base-docker-wsl/conf/env/create-env-linux-high-1.sh
    • devops/windows/base-docker-wsl/conf/env/create-env-linux-low-1.sh
    • devops/windows/base-docker-linux/conf/env/create-env-linux-low-1.sh
    • devops/windows/base-docker-linux/conf/env/create-env-linux-high-1.sh
    • devops/linux/docker/conf/env/create-env-linux-low-1.sh
    • devops/linux/docker/conf/env/create-env-wsl-low-1.sh
    • devops/linux/docker/conf/env/create-env-linux-high-1.sh

Please review and update the port configurations to ensure each service uses a unique port number.

Analysis chain

Verify the port configuration.

Setting all the services to the same port number (50000) may lead to port conflicts.

Run the following script to verify the port configuration:

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the port configuration of the API services.

# Test: Search for the port configuration of the API services. Expect: Each service should have a unique port number.
rg --type shell $'export .*_HTTP_PORT'

Length of output: 69


Script:

#!/bin/bash
# Description: Verify the port configuration of the API services.

# Test: Search for the port configuration of the API services in shell scripts.
rg --glob '*.sh' $'export .*_HTTP_PORT'

Length of output: 3974

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 1c56bef and 3664f10.

Files selected for processing (16)
  • JeMPI_Apps/JeMPI_API_KC/src/main/resources/config-api.json (1 hunks)
  • JeMPI_Apps/JeMPI_Bootstrapper/src/main/resources/config-api.json (1 hunks)
  • devops/windows/base-docker-linux/conf/env/conf-env-high-1-pc.template (7 hunks)
  • devops/windows/base-docker-linux/conf/env/conf-env-low-1-pc.template (1 hunks)
  • devops/windows/base-docker-linux/conf/env/create-env-linux-high-1.sh (1 hunks)
  • devops/windows/base-docker-linux/conf/env/create-env-linux-low-1.sh (3 hunks)
  • devops/windows/base-docker-linux/helper/scripts/d-stack-01-create-dirs.sh (1 hunks)
  • devops/windows/base-docker-wsl/conf/env/conf-env-high-1-pc.template (4 hunks)
  • devops/windows/base-docker-wsl/conf/env/conf-env-low-1-pc.template (1 hunks)
  • devops/windows/base-docker-wsl/conf/env/create-env-linux-high-1.sh (2 hunks)
  • devops/windows/base-docker-wsl/conf/env/create-env-linux-low-1.sh (3 hunks)
  • devops/windows/base-docker-wsl/helper/scripts/d-stack-01-create-dirs.sh (1 hunks)
  • devops/windows/deployment/deploy-local-windows.ps1 (4 hunks)
  • devops/windows/run--base-docker-linux/start.ps1 (1 hunks)
  • devops/windows/run--base-docker-wsl/start-with-bootstraper.ps1 (2 hunks)
  • devops/windows/run--base-docker-wsl/start.ps1 (2 hunks)
Additional context used
Biome
JeMPI_Apps/JeMPI_Bootstrapper/src/main/resources/config-api.json

[error] 1-1: String values must be double quoted.

(parse)


[error] 1-1: Minus must be followed by a digit

(parse)


[error] 1-1: String values must be double quoted.

(parse)


[error] 1-1: Minus must be followed by a digit

(parse)


[error] 1-1: String values must be double quoted.

(parse)


[error] 1-1: Minus must be followed by a digit

(parse)


[error] 1-1: String values must be double quoted.

(parse)


[error] 1-1: Minus must be followed by a digit

(parse)


[error] 1-1: String values must be double quoted.

(parse)


[error] 1-1: unexpected character .

(parse)


[error] 1-1: String values must be double quoted.

(parse)

Shellcheck
devops/windows/base-docker-linux/conf/env/create-env-linux-low-1.sh

[warning] 4-4: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 6-6: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 10-10: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 10-10: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)

devops/windows/base-docker-wsl/conf/env/create-env-linux-low-1.sh

[warning] 4-4: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 6-6: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 95-95: Declare and assign separately to avoid masking return values.

(SC2155)

devops/windows/base-docker-linux/conf/env/create-env-linux-high-1.sh

[warning] 4-4: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 6-6: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 16-16: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 18-18: Declare and assign separately to avoid masking return values.

(SC2155)

devops/windows/base-docker-wsl/conf/env/create-env-linux-high-1.sh

[warning] 4-4: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 6-6: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 16-16: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 18-18: Declare and assign separately to avoid masking return values.

(SC2155)

Gitleaks
devops/windows/base-docker-linux/conf/env/create-env-linux-low-1.sh

52-52: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)


54-54: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

devops/windows/base-docker-wsl/conf/env/create-env-linux-low-1.sh

52-52: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)


54-54: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

devops/windows/base-docker-linux/conf/env/create-env-linux-high-1.sh

65-65: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)


67-67: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

devops/windows/base-docker-wsl/conf/env/create-env-linux-high-1.sh

65-65: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)


67-67: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

Additional comments not posted (59)
devops/windows/base-docker-wsl/helper/scripts/d-stack-01-create-dirs.sh (1)

33-36: LGTM!

The code changes are approved for the following reasons:

  • The new directory creation commands are correctly implemented using the mkdir command with the -p flag.
  • The changes are consistent with the existing pattern of creating directories in the script.
  • The changes align with the AI-generated summary, which mentions the addition of directory creation commands for PostgreSQL backups, Dgraph backups, and configuration data management.
  • The changes do not alter any existing logic or conditions in the script.
devops/windows/base-docker-linux/helper/scripts/d-stack-01-create-dirs.sh (4)

36-38: LGTM!

Creating a dedicated directory for PostgreSQL database files and setting its ownership to a specific user and group is a good practice for better organization, management, and security.


38-38: LGTM!

Setting the permissions of the PostgreSQL database directory to 770 aligns with the principle of least privilege and helps protect sensitive database files by allowing full access only for the owner and group while restricting access for others.


40-40: LGTM!

Creating a dedicated directory for configuration files is a good practice for better organization and management.


42-43: LGTM!

Creating dedicated backup directories for PostgreSQL and Dgraph is a good practice for better organization and management of backup files.

devops/windows/base-docker-wsl/conf/env/conf-env-low-1-pc.template (1)

32-32: LGTM!

The addition of the new environment variable SYSTEM_CONFIG_DIR is a good practice as it improves the organization and accessibility of configuration files within the project structure.

devops/windows/base-docker-linux/conf/env/create-env-linux-low-1.sh (3)

5-5: LGTM!

The code changes are approved.


8-8: LGTM!

The code changes are approved.


31-35: LGTM!

The code changes are approved.

devops/windows/base-docker-wsl/conf/env/create-env-linux-low-1.sh (6)

4-4: LGTM!

The code changes are approved.

Tools
Shellcheck

[warning] 4-4: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


5-5: LGTM!

The code changes are approved.


8-8: LGTM!

The code changes are approved.


11-15: LGTM!

The code changes are approved.


31-35: LGTM!

The code changes are approved.


42-47: LGTM!

The code changes are approved.

devops/windows/base-docker-linux/conf/env/create-env-linux-high-1.sh (5)

5-5: LGTM!

The code changes are approved.


8-8: LGTM!

The code changes are approved.


11-14: LGTM!

The code changes are approved.


16-26: LGTM!

The code changes are approved.

Tools
Shellcheck

[warning] 16-16: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 18-18: Declare and assign separately to avoid masking return values.

(SC2155)


Line range hint 33-105: LGTM!

The code changes are approved.

Tools
Shellcheck

[warning] 4-4: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 6-6: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 16-16: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 18-18: Declare and assign separately to avoid masking return values.

(SC2155)

Gitleaks

65-65: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)


67-67: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

devops/windows/base-docker-wsl/conf/env/create-env-linux-high-1.sh (3)

5-5: LGTM!

The value of PROJECT_PATH_UI is set correctly using the previously exported PROJECT_PATH_APPS_ROOT variable.


8-8: LGTM!

The value of PROJECT_DATA_APPS_DIR is set correctly using the previously exported PROJECT_DIR variable.


11-14: LGTM!

The values of SYSTEM_CONFIG_DIR, API_CONFIG_REFERENCE_FILENAME, API_CONFIG_MASTER_FILENAME, and API_FIELDS_CONFIG_FILENAME are set correctly using string literals.

devops/windows/base-docker-linux/conf/env/conf-env-low-1-pc.template (1)

39-39: LGTM!

The code change is approved. The new environment variable SYSTEM_CONFIG_DIR improves the organization and accessibility of configuration resources by providing a dedicated path for system configuration files.

devops/windows/deployment/deploy-local-windows.ps1 (3)

13-13: LGTM!

The Java URL has been updated to version 21.0.3, ensuring that the script uses the latest Java version.


38-45: LGTM!

The new control flow structure improves the handling of WSL by checking if it is already running before attempting to start it. This prevents unnecessary restarts of WSL and enhances the user experience by displaying an informative message when WSL is already running.


13-13: Verify Java 21 compatibility in the codebase.

Before updating the Java version to 21.0.3, please ensure that the rest of the codebase is compatible with Java 21. This includes checking any third-party libraries, frameworks, or dependencies that may have specific Java version requirements.

Run the following script to search for Java version mentions in the codebase:

JeMPI_Apps/JeMPI_API_KC/src/main/resources/config-api.json (5)

2-21: LGTM!

The auxInteractionFields segment is well-structured and follows the expected schema. The code changes are approved.


22-39: LGTM!

The auxGoldenRecordFields segment is well-structured and follows the expected schema. The code changes are approved.


40-60: LGTM!

The additionalNodes segment is well-structured and follows the expected schema. The code changes are approved.


61-176: LGTM!

The demographicFields segment is well-structured and follows the expected schema. The indexing and linking metadata are properly configured. The code changes are approved.


177-208: LGTM!

The rules segment is well-structured and follows the expected schema. The deterministic and probabilistic rules are properly configured. The code changes are approved.

devops/windows/base-docker-linux/conf/env/conf-env-high-1-pc.template (13)

13-30: LGTM!

The new environment variable exports for API, backup-restore API, and ETL services enhance the service orchestration capabilities by providing a structured approach to manage the IP and port configurations.


37-40: LGTM!

The changes improve the organization and management of system and API configurations by:

  • Centralizing the system config directory in the project's DevOps directory.
  • Providing specific exports for different API config files (reference, master, and fields).

53-54: LGTM!

The new exports for Dgraph zero instance data directories (DATA_DGRAPH_ZERO_02_DIR and DATA_DGRAPH_ZERO_03_DIR) suggest an expansion of the Dgraph cluster, allowing for more distributed and scalable deployment scenarios.


59-61: LGTM!

The changes enhance the organization and management of PostgreSQL data and introduce dedicated directories for PostgreSQL and Dgraph backups, facilitating better data backup and recovery processes:

  • The DATA_POSTGRESQL_DIR export has been renamed to DATA_POSTGRESQL_DB_DIR for clarity.
  • New exports POSTGRES_BACKUP_DIRECTORY and DGRAPH_BACKUP_DIRECTORY have been added for dedicated backup directories.

68-68: Verify the impact of renaming the EM data directory to include "SCALA".

The renaming of the DATA_DIR_EM export to DATA_DIR_EM_SCALA suggests a shift towards using Scala for the EM component.

Please ensure that:

  1. The EM component has been updated to use Scala.
  2. The Scala-based EM component is compatible with the rest of the system.
  3. All relevant documentation and configuration files have been updated to reflect this change.

79-80: LGTM!

The addition of new exports for scaling Dgraph zero instances (SCALE_ZERO_02 and SCALE_ZERO_03) and the modifications to the exports for scaling Dgraph alpha instances (SCALE_ALPHA_02 and SCALE_ALPHA_03) suggest an expansion of the Dgraph cluster, allowing for more flexible and scalable deployment scenarios.

Also applies to: 82-83


92-92: This change is consistent with the previous renaming of the EM data directory export and has already been addressed in a previous comment.


101-102: LGTM!

The modifications to the Kafka and Dgraph placement exports indicate a shift towards a more distributed architecture, with Kafka and Dgraph instances being spread across multiple nodes:

  • PLACEMENT_KAFKA_02 and PLACEMENT_KAFKA_03 now reference NODE2 and NODE3 respectively.
  • PLACEMENT_DGRAPH_ZERO_02, PLACEMENT_DGRAPH_ZERO_03, PLACEMENT_DGRAPH_ALPHA_02, and PLACEMENT_DGRAPH_ALPHA_03 now reference NODE2 and NODE3.

This allows for better load balancing, fault tolerance, and scalability.

Also applies to: 104-105, 107-108


121-124: LGTM!

The addition of new exports for various PostgreSQL databases indicates an expansion of the database management capabilities, with dedicated databases for specific purposes:

  • POSTGRESQL_USERS_DB for user management
  • POSTGRESQL_NOTIFICATIONS_DB for notifications
  • POSTGRESQL_AUDIT_DB for auditing
  • POSTGRESQL_KC_TEST_DB for Keycloak testing

This segregation of databases promotes better organization, security, and performance.


139-139: This change is consistent with the previous renaming of the EM data directory and placement exports and has already been addressed in a previous comment.


146-147: LGTM!

The addition of new exports for API HTTP port (API_HTTP_PORT) and API Keycloak HTTP port (API_KC_HTTP_PORT) provides more granular control over the port configurations for the API and its integration with Keycloak, facilitating better customization and flexibility in deployment scenarios.


160-164: Verify that the removal of the conditional logic for setting the JeMPI base API port is intentional.

The commented-out code block suggests that the logic for setting the REACT_APP_JEMPI_BASE_API_PORT based on the value of REACT_APP_ENABLE_SSO is no longer needed or has been moved elsewhere.

Please ensure that:

  1. The removal of this conditional logic is intentional and does not break any existing functionality.
  2. The JeMPI base API port is correctly set elsewhere in the configuration or codebase.

167-167: LGTM!

The changes to the UI-related exports improve the configuration options for the frontend application, allowing for better customization and integration with backend services:

  • The renaming of REACT_APP_JEMPI_BASE_URL to REACT_APP_JEMPI_BASE_API_HOST provides a clearer distinction between the API host and port configurations.
  • The new exports for Keycloak frontend URL (KC_FRONTEND_URL), maximum upload CSV size (REACT_APP_MAX_UPLOAD_CSV_SIZE_IN_MEGABYTES), Node environment (NODE_ENV), and brand logo visibility (REACT_APP_SHOW_BRAND_LOGO) provide additional control over various aspects of the frontend application.

Also applies to: 171-174

devops/windows/base-docker-wsl/conf/env/conf-env-high-1-pc.template (5)

13-14: LGTM!

The new environment variables API_IP and API_HTTP_PORT improve the configuration capabilities for the API.


16-17: LGTM!

The new environment variables BACKUP_RESTORE_API_IP and BACKUP_RESTORE_API_HTTP_PORT improve the configuration capabilities for the backup and restore API.


19-20: LGTM!

The new environment variables improve the configuration capabilities for the API KC, linker, controller, and ETL components.

Also applies to: 22-23, 25-26, 28-29


37-40: LGTM!

The new environment variables improve the configuration capabilities for the system config directory and API config filenames.


53-54: LGTM!

The new environment variables improve the configuration capabilities for data directories, service placements, RAM limits, and API-related settings.

The commented-out code for conditionally setting the API port based on SSO enablement is a good addition for future use.

Also applies to: 63-72, 79-80, 88-97, 101-102, 104-105, 107-108, 112-116, 121-124, 135-143, 145-157, 160-164, 166-174

devops/windows/run--base-docker-linux/start.ps1 (3)

7-8: LGTM!

The changes to the file copy commands, updating the source of the config-reference-link-dp.json file and the destinations where it is copied as config-api.json, look good to me.


10-10: Looks good! Verify the correctness of ${SYSTEM_CONFIG_DIR}.

The new command to copy the config-reference-link-dp.json file to the system configuration directory specified by ${SYSTEM_CONFIG_DIR} looks good.

To ensure the correctness of the file copy operation, please verify that the ${SYSTEM_CONFIG_DIR} variable is correctly defined and points to the intended system configuration directory.

You can add the following line before the Copy-Item command to print the value of ${SYSTEM_CONFIG_DIR} for verification:

Write-Host "System config directory: ${SYSTEM_CONFIG_DIR}"

Line range hint 12-285: Remaining code looks good!

The remaining code in the script, including the variable definitions, configurations, and application startup commands, appears to be unchanged and looks good. No further changes are required.

devops/windows/run--base-docker-wsl/start.ps1 (1)

7-8: Verify the impact of using the new configuration file.

Ensure that the applications JeMPI_API and JeMPI_API_KC can correctly function with the settings defined in config-reference-link-dp.json.

Run the following script to verify the config usage:

#!/bin/bash
# Description: Verify if the `config-reference-link-dp.json` is being used.

# Test 1: Search for the old config file usage. Expect: No results.
rg --type-add 'configs:*.{json,yaml,yml,xml,config,conf,toml,ini,properties}' -g '!node_modules' -g '!.git' -g '!target' -g '!build' -g '!dist' -g '!venv' -g '!env' -g '!.idea' -g '!.vscode' -g '!.history' -g '!.pytest_cache' -g '!__pycache__' -g '!.mypy_cache' -g '!.coverage' -g '!htmlcov' -g '!.tox' -g '!.nox' -g '!.vagrant' -g '!.virtualenv' -g '!.env' -g '!.venv' -g '!.direnv' -g '!.envrc' -g '!.python-version' -g '!.tool-versions' -g '!.pre-commit-config.yaml' -g '!poetry.lock' -g '!Pipfile.lock' -g '!.terraform' -g '!.serverless' -g '!.sls' -g '!.serverless.yml' -g '!serverless.yml' -g '!.eslintcache' -g '!.eslintignore' -g '!.prettierignore' -g '!.prettierrc' -g '!.stylelintrc' -g '!.stylelintignore' -g '!.editorconfig' -g '!.nvmrc' -g '!.npmrc' -g '!.yarnrc' -g '!.yarn-integrity' -g '!.yarnclean' -g '!.yarn-error.log' -g '!.pnp.*' -g '!.DS_Store' -g '!.git' -g '!.hg' -g '!.svn' -g '!.idea' -g '!.vscode' -g '!*.sublime-project' -g '!*.sublime-workspace' -g '!*.code-workspace' -g '!.history' -g '!.pytest_cache' -g '!__pycache__' -g '!.mypy_cache' -g '!.coverage' -g '!htmlcov' -g '!.tox' -g '!.nox' -g '!.vagrant' -g '!.virtualenv' -g '!.env' -g '!.venv' -g '!.direnv' -g '!.envrc' -g '!.python-version' -g '!.tool-versions' -g '!.pre-commit-config.yaml' -g '!poetry.lock' -g '!Pipfile.lock' -g '!.terraform' -g '!.serverless' -g '!.sls' -g '!.serverless.yml' -g '!serverless.yml' -g '!.eslintcache' -g '!.eslintignore' -g '!.prettierignore' -g '!.prettierrc' -g '!.stylelintrc' -g '!.stylelintignore' -g '!.editorconfig' -g '!.nvmrc' -g '!.npmrc' -g '!.yarnrc' -g '!.yarn-integrity' -g '!.yarnclean' -g '!.yarn-error.log' -g '!.pnp.*' -g '!.DS_Store' -g '!.git' -g '!.hg' -g '!.svn' -g '!.idea' -g '!.vscode' -g '!*.sublime-project' -g '!*.sublime-workspace' -g '!*.code-workspace' -g '!.history' -g '!.pytest_cache' -g '!__pycache__' -g '!.mypy_cache' -g '!.coverage' -g '!htmlcov' -g '!.tox' -g '!.nox' -g '!.vagrant' -g '!.virtualenv' -g '!.env' -g '!.venv' -g '!.direnv' -g '!.envrc' -g '!.python-version' -g '!.tool-versions' -g '!.pre-commit-config.yaml' -g '!poetry.lock' -g '!Pipfile.lock' -g '!.terraform' -g '!.serverless' -g '!.sls' -g '!.serverless.yml' -g '!serverless.yml' -g '!.eslintcache' -g '!.eslintignore' -g '!.prettierignore' -g '!.prettierrc' -g '!.stylelintrc' -g '!.stylelintignore' -g '!.editorconfig' -g '!.nvmrc' -g '!.npmrc' -g '!.yarnrc' -g '!.yarn-integrity' -g '!.yarnclean' -g '!.yarn-error.log' -g '!.pnp.*' -g '!.DS_Store' -g '!.git' -g '!.hg' -g '!.svn' -g '!.idea' -g '!.vscode' -g '!*.sublime-project' -g '!*.sublime-workspace' -g '!*.code-workspace' -g '!.history' -g '!.pytest_cache' -g '!__pycache__' -g '!.mypy_cache' -g '!.coverage' -g '!htmlcov' -g '!.tox' -g '!.nox' -g '!.vagrant' -g '!.virtualenv' -g '!.env' -g '!.venv' -g '!.direnv' -g '!.envrc' -g '!.python-version' -g '!.tool-versions' -g '!.pre-commit-config.yaml' -g '!poetry.lock' -g '!Pipfile.lock' -g '!.terraform' -g '!.serverless' -g '!.sls' -g '!.serverless.yml' -g '!serverless.yml' -g '!.eslintcache' -g '!.eslintignore' -g '!.prettierignore' -g '!.prettierrc' -g '!.stylelintrc' -g '!.stylelintignore' -g '!.editorconfig' -g '!.nvmrc' -g '!.npmrc' -g '!.yarnrc' -g '!.yarn-integrity' -g '!.yarnclean' -g '!.yarn-error.log' -g '!.pnp.*' -g '!.DS_Store' -g '!.git' -g '!.hg' -g '!.svn' -g '!.idea' -g '!.vscode' -g '!*.sublime-project' -g '!*.sublime-workspace' -g '!*.code-workspace' -g '!.history' -g '!.pytest_cache' -g '!__pycache__' -g '!.mypy_cache' -g '!.coverage' -g '!htmlcov' -g '!.tox' -g '!.nox' -g '!.vagrant' -g '!.virtualenv' -g '!.env' -g '!.venv' -g '!.direnv' -g '!.envrc' -g '!.python-version' -g '!.tool-versions' -g '!.pre-commit-config.yaml' -g '!poetry.lock' -g '!Pipfile.lock' -g '!.terraform' -g '!.serverless' -g '!.sls' -g '!.serverless.yml' -g '!serverless.yml' -g '!.eslintcache' -g '!.eslintignore' -g '!.prettierignore' -g '!.prettierrc' -g '!.stylelintrc' -g '!.stylelintignore' -g '!.editorconfig' -g '!.nvmrc' -g '!.npmrc' -g '!.yarnrc' -g '!.yarn-integrity' -g '!.yarnclean' -g '!.yarn-error.log' -g '!.pnp.*' -g '!.DS_Store' -g '!.git' -g '!.hg' -g '!.svn' -g '!.idea' -g '!.vscode' -g '!*.sublime-project' -g '!*.sublime-workspace' -g '!*.code-workspace' -g '!.history' -g '!.pytest_cache' -g '!__pycache__' -g '!.mypy_cache' -g '!.coverage' -g '!htmlcov' -g '!.tox' -g '!.nox' -g '!.vagrant' -g '!.virtualenv' -g '!.env' -g '!.venv' -g '!.direnv' -g '!.envrc' -g '!.python-version' -g '!.tool-versions' -g '!.pre-commit-config.yaml' -g '!poetry.lock' -g '!Pipfile.lock' -g '!.terraform' -g '!.serverless' -g '!.sls' -g '!.serverless.yml' -g '!serverless.yml' -g '!.eslintcache' -g '!.eslintignore' -g '!.prettierignore' -g '!.prettierrc' -g '!.stylelintrc' -g '!.stylelintignore' -g '!.editorconfig' -g '!.nvmrc' -g '!.npmrc' -g '!.yarnrc' -g '!.yarn-integrity' -g '!.yarnclean' -g '!.yarn-error.log' -g '!.pnp.*' -g '!.DS_Store' -g '!.git' -g '!.hg' -g '!.svn' -g '!.idea' -g '!.vscode' -g '!*.sublime-project' -g '!*.sublime-workspace' -g '!*.code-workspace' -g '!.history' -g '!.pytest_cache' -g '!__pycache__' -g '!.mypy_cache' -g '!.coverage' -g '!htmlcov' -g '!.tox' -g '!.nox' -g '!.vagrant' -g '!.virtualenv' -g '!.env' -g '!.venv' -g '!.direnv' -g '!.envrc' -g '!.python-version' -g '!.tool-versions' -g '!.pre-commit-config.yaml' -g '!poetry.lock' -g '!Pipfile.lock' -g '!.terraform' -g '!.serverless' -g '!.sls' -g '!.serverless.yml' -g '!serverless.yml' -g '!.eslintcache' -g '!.eslintignore' -g '!.prettierignore' -g '!.prettierrc' -g '!.stylelintrc' -g '!.stylelintignore' -g '!.editorconfig' -g '!.nvmrc' -g '!.npmrc' -g '!.yarnrc' -g '!.yarn-integrity' -g '!.yarnclean' -g '!.yarn-error.log' -g '!.pnp.*' -g '!.DS_Store' -g '!.git' -g '!.hg' -g '!.svn' -g '!.idea' -g '!.vscode' -g '!*.sublime-project' -g '!*.sublime-workspace' -g '!*.code-workspace' -g '!.history' -g '!.pytest_cache' -g '!__pycache__' -g '!.mypy_cache' -g '!.coverage' -g '!htmlcov' -g '!.tox' -g '!.nox' -g '!.vagrant' -g '!.virtualenv' -g '!.env' -g '!.venv' -g '!.direnv' -g '!.envrc' -g '!.python-version' -g '!.tool-versions' -g '!.pre-commit-config.yaml' -g '!poetry.lock' -g '!Pipfile.lock' -g '!.terraform' -g '!.serverless' -g '!.sls' -g '!.serverless.yml' -g '!serverless.yml' -g '!.eslintcache' -g '!.eslintignore' -g '!.prettierignore' -g '!.prettierrc' -g '!.stylelintrc' -g '!.stylelintignore' -g '!.editorconfig' -g '!.nvmrc' -g '!.npmrc' -g '!.yarnrc' -g '!.yarn-integrity' -g '!.yarnclean' -g '!.yarn-error.log' -g '!.pnp.*' -g '!.DS_Store' -g '!.git' -g '!.hg' -g '!.svn' -g '!.idea' -g '!.vscode' -g '!*.sublime-project' -g '!*.sublime-workspace' -g '!*.code-workspace' -g '!.history' -g '!.pytest_cache' -g '!__pycache__' -g '!.mypy_cache' -g '!.coverage' -g '!htmlcov' -g '!.tox' -g '!.nox' -g '!.vagrant' -g '!.virtualenv' -g '!.env' -g '!.venv' -g '!.direnv' -g '!.envrc' -g '!.python-version' -g '!.tool-versions' -g '!.pre-commit-config.yaml' -g '!poetry.lock' -g '!Pipfile.lock' -g '!.terraform' -g '!.serverless' -g '!.sls' -g '!.serverless.yml' -g '!serverless.yml' -g '!.eslintcache' -g '!.eslintignore' -g '!.prettierign

</blockquote></details>
<details>
<summary>devops/windows/run--base-docker-wsl/start-with-bootstraper.ps1 (5)</summary><blockquote>

`4-5`: **LGTM!**

The logging statements enhance the script's output clarity during execution.

---

Line range hint `1-282`: **LGTM!**

The remaining code maintains its existing functionality with minor adjustments to the paths used for copying files.

---

`11-11`: **Verify the usage of the configuration file in the new location.**

Ensure that the configuration file `config-reference-link-dp.json` placed at `${SYSTEM_CONFIG_DIR}` is being used correctly.

Run the following script to verify the configuration file usage:

---

`104-110`: **Verify the impact of the configuration file changes.**

Ensure that the applications `JeMPI_API` and `JeMPI_Bootstrapper` can handle the updated configuration file `config-api.json`.

Run the following script to verify the configuration file usage:

---

`8-9`: **Verify the impact of the configuration file changes.**

Ensure that the applications `JeMPI_API` and `JeMPI_API_KC` can handle the updated configuration file `config-reference-link-dp.json`.

Run the following script to verify the configuration file usage:

</blockquote></details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 13

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 3664f10 and 0185543.

Files selected for processing (19)
  • .gitignore (1 hunks)
  • JeMPI_Apps/JeMPI_API_KC/src/main/resources/config-api.json (1 hunks)
  • JeMPI_Apps/JeMPI_EM_Scala/src/main/scala/org/jembi/jempi/em/EM_Scala.scala (3 hunks)
  • JeMPI_Apps/JeMPI_LibAPI/src/main/java/org/jembi/jempi/libapi/BackEnd.java (2 hunks)
  • JeMPI_Apps/JeMPI_LibShared/src/main/java/org/jembi/jempi/shared/config/Config.java (2 hunks)
  • devops/windows/base-docker-linux/conf/env/conf-env-high-1-pc.template (7 hunks)
  • devops/windows/base-docker-linux/conf/env/conf-env-low-1-pc.template (1 hunks)
  • devops/windows/base-docker-linux/conf/env/create-env-linux-high-1.sh (1 hunks)
  • devops/windows/base-docker-linux/conf/env/create-env-linux-low-1.sh (3 hunks)
  • devops/windows/base-docker-linux/helper/scripts/d-stack-01-create-dirs.sh (1 hunks)
  • devops/windows/base-docker-wsl/conf/env/conf-env-high-1-pc.template (4 hunks)
  • devops/windows/base-docker-wsl/conf/env/conf-env-low-1-pc.template (1 hunks)
  • devops/windows/base-docker-wsl/conf/env/create-env-linux-high-1.sh (1 hunks)
  • devops/windows/base-docker-wsl/conf/env/create-env-linux-low-1.sh (3 hunks)
  • devops/windows/base-docker-wsl/helper/scripts/d-stack-01-create-dirs.sh (1 hunks)
  • devops/windows/deployment/deploy-local-windows.ps1 (4 hunks)
  • devops/windows/run--base-docker-linux/start.ps1 (3 hunks)
  • devops/windows/run--base-docker-wsl/start-with-bootstraper.ps1 (4 hunks)
  • devops/windows/run--base-docker-wsl/start.ps1 (5 hunks)
Files skipped from review due to trivial changes (1)
  • .gitignore
Files skipped from review as they are similar to previous changes (8)
  • devops/windows/base-docker-linux/conf/env/conf-env-high-1-pc.template
  • devops/windows/base-docker-linux/helper/scripts/d-stack-01-create-dirs.sh
  • devops/windows/base-docker-wsl/conf/env/conf-env-low-1-pc.template
  • devops/windows/base-docker-wsl/helper/scripts/d-stack-01-create-dirs.sh
  • devops/windows/deployment/deploy-local-windows.ps1
  • devops/windows/run--base-docker-linux/start.ps1
  • devops/windows/run--base-docker-wsl/start-with-bootstraper.ps1
  • devops/windows/run--base-docker-wsl/start.ps1
Additional context used
Shellcheck
devops/windows/base-docker-linux/conf/env/create-env-linux-low-1.sh

[warning] 4-4: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 6-6: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 10-10: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 10-10: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)

devops/windows/base-docker-wsl/conf/env/create-env-linux-low-1.sh

[warning] 4-4: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 6-6: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 95-95: Declare and assign separately to avoid masking return values.

(SC2155)

devops/windows/base-docker-linux/conf/env/create-env-linux-high-1.sh

[warning] 4-4: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 6-6: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 16-16: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 18-18: Declare and assign separately to avoid masking return values.

(SC2155)

devops/windows/base-docker-wsl/conf/env/create-env-linux-high-1.sh

[warning] 4-4: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 6-6: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 16-16: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 18-18: Declare and assign separately to avoid masking return values.

(SC2155)

Gitleaks
devops/windows/base-docker-linux/conf/env/create-env-linux-low-1.sh

52-52: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)


54-54: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

devops/windows/base-docker-wsl/conf/env/create-env-linux-low-1.sh

52-52: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)


54-54: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

devops/windows/base-docker-linux/conf/env/create-env-linux-high-1.sh

65-65: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)


67-67: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

devops/windows/base-docker-wsl/conf/env/create-env-linux-high-1.sh

65-65: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)


67-67: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

Additional comments not posted (5)
devops/windows/base-docker-linux/conf/env/conf-env-low-1-pc.template (1)

21-24: Approve new environment variable declarations.

The new environment variable declarations enhance configuration management and are correctly implemented. However, ensure that these variables are used consistently throughout the project.

Run the following script to verify the usage of the new environment variables across the project:

Verification successful

Environment variables are consistently used across the project.

The newly declared environment variables are utilized in various scripts, configuration files, and source code, indicating consistent usage throughout the project.

  • PowerShell scripts: devops/windows/run--base-docker-wsl/start-with-bootstraper.ps1, devops/windows/run--base-docker-linux/start.ps1
  • YAML configuration files: devops/linux/docker/conf/stack/docker-stack-low-1.yml, devops/linux/docker/conf/stack/docker-stack-high-1.yml
  • Java source files: JeMPI_Apps/JeMPI_API/src/main/java/org/jembi/jempi/AppConfig.java, JeMPI_Apps/JeMPI_API/src/main/java/org/jembi/jempi/api/API.java
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the usage of new environment variables across the project.

# Test: Search for the usage of new environment variables. Expect: Consistent usage across the project.
rg --type bash $'SYSTEM_CONFIG_DIR|API_CONFIG_REFERENCE_FILENAME|API_CONFIG_MASTER_FILENAME|API_FIELDS_CONFIG_FILENAME'

Length of output: 149


Script:

#!/bin/bash
# Description: Verify the usage of new environment variables across the project.

# Test: Search for the usage of new environment variables across all files.
rg 'SYSTEM_CONFIG_DIR|API_CONFIG_REFERENCE_FILENAME|API_CONFIG_MASTER_FILENAME|API_FIELDS_CONFIG_FILENAME'

Length of output: 20051

JeMPI_Apps/JeMPI_EM_Scala/src/main/scala/org/jembi/jempi/em/EM_Scala.scala (2)

43-43: Approved: System-dependent file separator usage.

The introduction of the separator variable using FileSystems.getDefault().getSeparator() is a robust way to handle file path separators across different operating systems.


64-64: Approved: Use of dynamic file path in jsonMapper.readValue.

Updating the jsonMapper.readValue method to use the dynamically determined filePath enhances the flexibility and adaptability of the application's configuration management.

devops/windows/base-docker-wsl/conf/env/conf-env-high-1-pc.template (2)

13-26: Approved: Introduction of API-related environment variables.

The new environment variables for API configurations are well-defined and follow common practices for Docker environments. Ensure that the placeholders like ${API_HTTP_PORT} are correctly set in the deployment scripts to avoid runtime issues.

Verify that the deployment scripts correctly set these placeholders:

Verification successful

Verified: Placeholders for API-related environment variables are correctly set.

The placeholders for API_HTTP_PORT and other related environment variables are being used and set appropriately in various scripts and configuration files. This aligns with common practices for managing environment variables in deployment scenarios. The deployment scripts appear to handle these variables correctly, addressing the review comment's concern.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that placeholders are correctly set in deployment scripts.

# Test: Search for placeholder usage in deployment scripts. Expect: Correct assignments.
rg --type shell -A 5 $'API_HTTP_PORT'

Length of output: 66


Script:

#!/bin/bash
# Description: Verify that placeholders are correctly set in deployment scripts.

# Test: Search for placeholder usage across all files. Expect: Correct assignments.
rg 'API_HTTP_PORT' -A 5

Length of output: 28065


87-107: Approved: Strategic distribution of services across nodes.

The modifications and introductions of environment variables for service placements are well-thought-out, aiming to optimize resource distribution and load balancing. Ensure that the nodes are correctly assigned in the deployment configuration to reflect these changes effectively.

Verify that the nodes are correctly assigned in the deployment configuration:

Verification successful

Node Assignments Verified: Correct Usage in Deployment Configuration

The node variables NODE1, NODE2, and NODE3 are correctly assigned and utilized across various configuration and script files. This confirms the strategic distribution of services across nodes for load balancing and resource optimization is implemented as intended. No further action is required.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify node assignments in deployment configuration.

# Test: Search for node usage in deployment configuration. Expect: Correct assignments.
rg --type shell -A 5 $'NODE1'

Length of output: 58


Script:

#!/bin/bash
# Description: Verify node assignments in deployment configuration.

# Test: Search for node usage in all files. Expect: Correct assignments.
rg 'NODE[123]' -A 5

Length of output: 57957

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 12

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 0185543 and 77c883e.

Files selected for processing (13)
  • .gitignore (1 hunks)
  • JeMPI_Apps/JeMPI_LibAPI/src/main/java/org/jembi/jempi/libapi/BackEnd.java (3 hunks)
  • devops/windows/base-docker-linux/conf/env/conf-env-high-1-pc.template (7 hunks)
  • devops/windows/base-docker-linux/conf/env/conf-env-low-1-pc.template (1 hunks)
  • devops/windows/base-docker-linux/conf/env/create-env-linux-high-1.sh (1 hunks)
  • devops/windows/base-docker-linux/conf/env/create-env-linux-low-1.sh (3 hunks)
  • devops/windows/base-docker-linux/helper/scripts/d-stack-01-create-dirs.sh (1 hunks)
  • devops/windows/base-docker-wsl/conf/env/conf-env-high-1-pc.template (4 hunks)
  • devops/windows/base-docker-wsl/conf/env/conf-env-low-1-pc.template (1 hunks)
  • devops/windows/base-docker-wsl/conf/env/create-env-linux-high-1.sh (1 hunks)
  • devops/windows/base-docker-wsl/conf/env/create-env-linux-low-1.sh (3 hunks)
  • devops/windows/base-docker-wsl/helper/scripts/d-stack-01-create-dirs.sh (1 hunks)
  • devops/windows/run--base-docker-wsl/start.ps1 (4 hunks)
Files skipped from review due to trivial changes (2)
  • devops/windows/base-docker-linux/conf/env/conf-env-low-1-pc.template
  • devops/windows/base-docker-wsl/conf/env/conf-env-low-1-pc.template
Files skipped from review as they are similar to previous changes (5)
  • .gitignore
  • devops/windows/base-docker-linux/conf/env/conf-env-high-1-pc.template
  • devops/windows/base-docker-linux/helper/scripts/d-stack-01-create-dirs.sh
  • devops/windows/base-docker-wsl/helper/scripts/d-stack-01-create-dirs.sh
  • devops/windows/run--base-docker-wsl/start.ps1
Additional context used
Shellcheck
devops/windows/base-docker-linux/conf/env/create-env-linux-low-1.sh

[warning] 4-4: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 6-6: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 10-10: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 10-10: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)

devops/windows/base-docker-wsl/conf/env/create-env-linux-low-1.sh

[warning] 4-4: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 6-6: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 96-96: Declare and assign separately to avoid masking return values.

(SC2155)

devops/windows/base-docker-linux/conf/env/create-env-linux-high-1.sh

[warning] 4-4: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 6-6: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 17-17: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 19-19: Declare and assign separately to avoid masking return values.

(SC2155)

devops/windows/base-docker-wsl/conf/env/create-env-linux-high-1.sh

[warning] 4-4: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 6-6: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 6-6: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


[warning] 17-17: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 19-19: Declare and assign separately to avoid masking return values.

(SC2155)

Gitleaks
devops/windows/base-docker-linux/conf/env/create-env-linux-low-1.sh

53-53: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)


55-55: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

devops/windows/base-docker-wsl/conf/env/create-env-linux-low-1.sh

53-53: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)


55-55: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

devops/windows/base-docker-linux/conf/env/create-env-linux-high-1.sh

66-66: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)


68-68: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

devops/windows/base-docker-wsl/conf/env/create-env-linux-high-1.sh

66-66: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)


68-68: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

Additional comments not posted (17)
devops/windows/base-docker-linux/conf/env/create-env-linux-low-1.sh (2)

8-8: Path declaration for application data is clear and straightforward.

The declaration of PROJECT_DATA_APPS_DIR is straightforward and does not involve any risky operations. This change is approved.


78-78: Resource limit for Scala environment is appropriately set.

The setting of EM_SCALA_RAM_LIMIT to "16G" is clear and appropriate for managing resource limits within the environment. This change is approved.

devops/windows/base-docker-wsl/conf/env/create-env-linux-high-1.sh (7)

34-35: Approved: Scaling configuration changes.

The changes to SCALE_ZERO_02 and SCALE_ZERO_03 are straightforward and correctly set the scaling configuration to 0.


41-41: Approved: Updated scaling configuration for linker.

The change to SCALE_LINKER correctly updates the scaling configuration to 3, aligning with the intended scaling enhancements.


45-49: Approved: Updated PostgreSQL database configurations.

The changes to the PostgreSQL database environment variables, including the update from a specific database name to postgres, correctly reflect a more structured approach to database management.


56-61: Approved: Uniform HTTP port configuration for services.

Setting all service HTTP ports to 50000 is a straightforward change that aligns with the intended restructuring of service endpoints.


77-81: Approved: UI configuration settings.

The changes to the UI configuration, including the API host and port settings, are straightforward and correctly set.


87-96: Approved: Optimized RAM limits for services.

The changes to the RAM limits for various services, including decreases and new additions, correctly reflect the intended optimization of resource allocation.


98-100: Approved: Additional UI configuration settings.

The changes to the UI configuration, including the base URL and backend mock settings, are straightforward and correctly set.

devops/windows/base-docker-wsl/conf/env/create-env-linux-low-1.sh (3)

8-8: Approved: New data directory environment variable.

The introduction of the PROJECT_DATA_APPS_DIR environment variable is well-executed. The naming convention is clear and consistent, aiding in the organization of data storage.


82-89: Approved: New RAM limit environment variables.

The introduction of RAM limit variables for various components such as ASYNC_RECEIVER_RAM_LIMIT, ETL_RAM_LIMIT, and others is well-executed. These settings are crucial for managing resource allocation effectively in a Docker environment.


32-32: Verify the impact of changing the primary database name.

The modification of the POSTGRESQL_DATABASE variable from "notifications" to "postgres" could have significant implications on the system's database interactions. Ensure that all references to this database are updated accordingly and that the change is reflected in all relevant parts of the system.

Verification successful

Change to POSTGRESQL_DATABASE is correctly implemented.

The modification of the POSTGRESQL_DATABASE variable from "notifications" to "postgres" does not have any unresolved references in the codebase. The occurrences of "notifications" found are related to different variables and contexts, which do not impact this change. No further action is required.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all references to the `POSTGRESQL_DATABASE` are updated.

# Test: Search for the old database name in the codebase. Expect: No occurrences of the old name.
rg --type bash --type python --type sql "notifications"

Length of output: 84


Script:

#!/bin/bash
# Description: Verify all references to the `POSTGRESQL_DATABASE` are updated.

# Test: Search for the old database name in the codebase. Expect: No occurrences of the old name.
rg --type sh --type py --type sql "notifications"

Length of output: 1419

devops/windows/base-docker-linux/conf/env/create-env-linux-high-1.sh (3)

4-4: Skip: Existing comment still valid.

The previous comment about improving error handling for directory navigation failures is still applicable.

Tools
Shellcheck

[warning] 4-4: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 4-4: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.

(SC2164)


17-19: Skip: Existing comment still valid.

The previous comment about enhancing clarity and error handling for node configuration is still applicable.

Tools
Shellcheck

[warning] 17-17: Declare and assign separately to avoid masking return values.

(SC2155)


[warning] 19-19: Declare and assign separately to avoid masking return values.

(SC2155)


66-66: Skip: Existing comment still valid.

The previous comment about removing sensitive information from the script is still applicable.

Tools
Gitleaks

66-66: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.

(generic-api-key)

devops/windows/base-docker-wsl/conf/env/conf-env-high-1-pc.template (2)

13-15: Approved: New API environment variables introduced.

The introduction of API_IP and API_HTTP_PORT enhances API connectivity and management. Consider adding documentation to describe the intended use of these variables, enhancing clarity and maintainability.


101-102: Approved: Strategic distribution of Kafka services.

The update to PLACEMENT_KAFKA_02 and PLACEMENT_KAFKA_03 to use different nodes (NODE2 and NODE3) is a strategic choice that may enhance fault tolerance and resource distribution. Monitor the impact on service performance and stability.

export JEMPI_SESSION_SECURE="false"
export JEMPI_SESSION_DOMAIN_NAME="localhost"

# UI env vars
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct the typo in NODE_ENV and approve the production setting.

The setting of NODE_ENV to "production" is approved as it aligns with standard practices for production environments. However, there is a typo in a later part of the script where "development" is misspelled as "developement". This should be corrected to maintain consistency and avoid potential configuration errors.

Apply this diff to fix the typo:

-export NODE_ENV="developement"
+export NODE_ENV="development"

Also applies to: 92-92

Copy link
Collaborator

@NyashaMuusha NyashaMuusha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

devops/windows/base-docker-wsl/conf/env/create-env-linux-low-1.sh
line 92,
-export NODE_ENV="developement"
change to
+export NODE_ENV="development"

@MatthewErispe MatthewErispe merged commit 06c0a27 into dev Sep 11, 2024
1 check passed
@MatthewErispe MatthewErispe deleted the CU-86c029q15_Update-Windows-Installation-Scripts branch September 11, 2024 13:00
@coderabbitai coderabbitai bot mentioned this pull request Sep 12, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants