feat(api): Add support for storing and managing variables

[rajdip-b]

User description

Fixes #91

---

Type

enhancement, tests

---

Description

• Introduced the VariableModule for managing variables within the application.
• Added CRUD operations for variables including version management.
• Extended event handling to support variable-related events.
• Implemented utility functions for fetching variables with authority checks and default project environments.
• Added comprehensive tests covering new variable functionality and related event handling.

---

Changes walkthrough

	Relevant files
Enhancement	13 files app.module.ts Integrate Variable Module into App Module                                apps/api/src/app/app.module.ts Imported and added VariableModule to the application module imports. +3/-1      cleanup.ts Add Variable Cleanup Logic                                                              apps/api/src/common/cleanup.ts Added cleanup logic for prisma.variable.deleteMany() to remove all variables during cleanup. +2/-1      create-event.ts Extend Event Creation for Variable Entity                                apps/api/src/common/create-event.ts Extended createEvent function to handle Variable entity events. Added case for EventSource.VARIABLE to assign sourceVariableId. +16/-2    get-default-project-environemnt.ts Utility to Fetch Default Project Environment                          apps/api/src/common/get-default-project-environemnt.ts Added a new utility function to fetch the default environment of a project. +13/-0    get-variable-with-authority.ts Fetch Variable with Authority Verification                              apps/api/src/common/get-variable-with-authority.ts Implemented a function to fetch a variable with authority checks. +57/-0    event.controller.ts Extend Event Controller to Support Variable Events              apps/api/src/event/controller/event.controller.ts Extended EventController to include variableId in query parameters for fetching events. +2/-0      event.service.ts Support Variable ID in Event Service                                          apps/api/src/event/service/event.service.ts Extended EventService to handle variable ID in event fetching logic. +10/-0    secret.service.ts Refactor and Extend Secret Service with Events                      apps/api/src/secret/service/secret.service.ts Refactored to use getDefaultEnvironmentOfProject utility. Added event creation for secret rollback and deletion actions. +40/-14  variable.controller.ts Implement Variable Controller                                                        apps/api/src/variable/controller/variable.controller.ts Implemented CRUD operations for variables in VariableController. +113/-0  create.variable.ts Define CreateVariable DTO                                                                apps/api/src/variable/dto/create.variable/create.variable.ts Defined CreateVariable DTO with validation decorators. +13/-0    variable.service.ts Implement Variable Service                                                              apps/api/src/variable/service/variable.service.ts Implemented VariableService with methods for CRUD operations and version management. +464/-0  variable.module.ts Define Variable Module                                                                      apps/api/src/variable/variable.module.ts Defined VariableModule with controller and service providers. +9/-0      variable.types.ts Define Types for Variable Entities                                              apps/api/src/variable/variable.types.ts Defined types for variable-related entities. +16/-0
Tests	6 files event.e2e.spec.ts E2E Tests for Variable Events                                                        apps/api/src/event/event.e2e.spec.ts Added end-to-end tests for event functionality involving variables. Included VariableService and VariableModule in test setup. Added test case for fetching a variable event. +43/-1    variable.controller.spec.ts Unit Tests for Variable Controller                                              apps/api/src/variable/controller/variable.controller.spec.ts Added unit tests for VariableController. +30/-0    create.variable.spec.ts Unit Tests for CreateVariable DTO                                                apps/api/src/variable/dto/create.variable/create.variable.spec.ts Added unit tests for CreateVariable DTO. +7/-0      update.variable.spec.ts Unit Tests for UpdateVariable DTO                                                apps/api/src/variable/dto/update.variable/update.variable.spec.ts Added unit tests for UpdateVariable DTO. +7/-0      variable.service.spec.ts Unit Tests for Variable Service                                                    apps/api/src/variable/service/variable.service.spec.ts Added unit tests for VariableService. +28/-0    variable.e2e.spec.ts E2E Tests for Variable Functionality                                          apps/api/src/variable/variable.e2e.spec.ts Added end-to-end tests for variable functionality. +865/-0
Configuration changes	2 files migration.sql Database Migration for Variable Entities                                  apps/api/src/prisma/migrations/20240219163241_add_variable/migration.sql Migration script to add variable and variable version entities to the database schema. +91/-0    schema.prisma Update Prisma Schema for Variable Models                                  apps/api/src/prisma/schema.prisma Updated Prisma schema to include variable and variable version models. +59/-6

---

✨ PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

[sonarcloud]

Quality Gate passed

Issues
0 New issues

Measures
0 Security Hotspots
No data about Coverage
28.1% Duplication on New Code

See analysis details on SonarCloud

[codiumai-pr-agent-free]

PR Description updated to latest commit (b61803a)

[codiumai-pr-agent-free]

PR Review

PR feedback
⏱️ Estimated effort to review [1-5]	4, because the PR introduces a significant amount of new functionality across multiple files, including a new module, service, controller, DTOs, and integration with existing systems like events and permissions. The complexity is increased by the introduction of variable versioning and environment-specific logic.
🧪 Relevant tests	Yes
🔍 Possible issues	There is a potential issue with the unique constraint on variables (Variable_projectId_environmentId_name_key). If a variable is moved between environments within the same project, and a variable with the same name exists in the target environment, the operation will fail. This scenario seems to be handled in the code, but it's a complex logic that could be prone to errors. The rollback functionality for variables does not check if the rollback version is less than the current version. This could potentially allow rolling back to a version higher than the current one, which doesn't make sense in the context of a rollback operation. The VariableService directly accesses the Prisma client for database operations, which is fine, but it increases the coupling between the service and the Prisma client. Consider abstracting these operations behind a repository layer to make the service easier to test and maintain.
🔒 Security concerns	No

---

✨ Review tool usage guide:

---

Overview:
The review tool scans the PR code changes, and generates a PR review. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.
When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:

/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

With a configuration file, use the following template:

[pr_reviewer]
some_config1=...
some_config2=...

Utilizing extra instructions The review tool can be configured with extra instructions, which can be used to guide the model to a feedback tailored to the needs of your project. Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify the relevant sub-tool, and the relevant aspects of the PR that you want to emphasize. Examples for extra instructions: [pr_reviewer] # /review # extra_instructions=""" In the 'possible issues' section, emphasize the following: - Does the code logic cover relevant edge cases? - Is the code logic clear and easy to understand? - Is the code logic efficient? ... """ Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

How to enable\disable automation When you first install PR-Agent app, the default mode for the review tool is: pr_commands = ["/review", ...] meaning the review tool will run automatically on every PR, with the default configuration. Edit this field to enable/disable the tool, or to change the used configurations

Auto-labels The review tool can auto-generate two specific types of labels for a PR: a possible security issue label, that detects possible security issues (enable_review_labels_security flag) a Review effort [1-5]: x label, where x is the estimated effort to review the PR (enable_review_labels_effort flag)

Extra sub-tools The review tool provides a collection of possible feedbacks about a PR. It is recommended to review the possible options, and choose the ones relevant for your use case. Some of the feature that are disabled by default are quite useful, and should be considered for enabling. For example: require_score_review, require_soc2_ticket, and more.

Auto-approve PRs By invoking: /review auto_approve The tool will automatically approve the PR, and add a comment with the approval. To ensure safety, the auto-approval feature is disabled by default. To enable auto-approval, you need to actively set in a pre-defined configuration file the following: [pr_reviewer] enable_auto_approval = true (this specific flag cannot be set with a command line argument, only in the configuration file, committed to the repository) You can also enable auto-approval only if the PR meets certain requirements, such as that the estimated_review_effort is equal or below a certain threshold, by adjusting the flag: [pr_reviewer] maximal_review_effort = 5

More PR-Agent commands To invoke the PR-Agent, add a comment using one of the following commands: /review: Request a review of your Pull Request. /describe: Update the PR title and description based on the contents of the PR. /improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback. /ask <QUESTION>: Ask a question about the PR. /update_changelog: Update the changelog based on the PR's contents. /add_docs 💎: Generate docstring for new components introduced in the PR. /generate_labels 💎: Generate labels for the PR based on the PR's contents. /analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component. See the tools guide for more details. To list the possible configuration parameters, add a /config comment.

See the review usage page for a comprehensive guide on using this tool.

[codiumai-pr-agent-free]

PR Code Suggestions

	Suggestions
enhancement	Improve error handling in the catch block.                                    Consider implementing a more specific error handling strategy instead of an empty catch block. This will help in identifying and debugging issues more effectively. apps/api/src/common/get-variable-with-authority.ts [31-33] } catch (error) { - /* empty */ + console.error(`Failed to fetch variable with ID ${variableId}:`, error); + throw new InternalServerErrorException(`Error fetching variable with ID ${variableId}`); } Use a transaction for the rollback operation.                                 To ensure the rollback operation is atomic and to handle potential errors more gracefully, consider wrapping the deletion and event creation logic inside a transaction. apps/api/src/secret/service/secret.service.ts [314-321] -const result = await this.prisma.secretVersion.deleteMany({ - where: { - secretId, - version: { - gt: Number(rollbackVersion) +const result = await this.prisma.$transaction(async (prisma) => { + const deleteResult = await prisma.secretVersion.deleteMany({ + where: { + secretId, + version: { + gt: Number(rollbackVersion) + } } - } -}) + }); + await createEvent({...}, prisma); + return deleteResult; +}); Wrap deletion operations in a transaction for atomicity.                      To ensure that all deletions are performed as a single atomic operation, consider wrapping the deletion operations in a transaction. apps/api/src/common/cleanup.ts [11-15] -await Promise.all([ +await prisma.$transaction([ prisma.project.deleteMany(), prisma.user.deleteMany(), prisma.event.deleteMany(), prisma.apiKey.deleteMany(), prisma.variable.deleteMany() ]) Use transactions for multiple database operations to ensure data consistency. Consider using a transaction for operations that involve multiple steps or queries to ensure data consistency. This is particularly useful in functions like createVariable, updateVariable, and deleteVariable where multiple database operations are performed. NestJS's PrismaService can facilitate transactions using the prisma.$transaction method. apps/api/src/variable/service/variable.service.ts [76-104] -const variable = await this.prisma.variable.create({ - data: { - name: dto.name, - versions: { - create: { - value: dto.value, - version: 1, - createdBy: { - connect: { - id: user.id - } - } - } - }, - environment: { - connect: { - id: environment.id - } - }, - project: { - connect: { - id: projectId - } - }, - lastUpdatedBy: { - connect: { - id: user.id - } - } - } -}) +const variable = await this.prisma.$transaction(async (prisma) => { + // Wrap your operations here +}); Enhance error messages for better debugging and error handling.               For better error handling and debugging, consider adding more specific error messages in the NotFoundException and ConflictException throughout your service methods. This will help in quickly identifying issues related to specific entities. apps/api/src/variable/service/variable.service.ts [63-65] throw new NotFoundException( - `No default environment found for project with id ${projectId}` + `No default environment found for project with id ${projectId}. Ensure the project exists and has a default environment set.` ) Add a description for the new enum value to clarify its purpose.              Consider adding a description for the VARIABLE enum value in EventSource to clarify its purpose and usage within the system. apps/api/src/prisma/schema.prisma [12] -VARIABLE +VARIABLE // Description of VARIABLE's purpose Add an isActive field to VariableVersion for better version management.       Ensure that the VariableVersion model includes a field or mechanism to track whether a version is active or has been superseded by a newer version, to facilitate version management and rollback. apps/api/src/prisma/schema.prisma [337-340] model VariableVersion { id String @id @default(cuid()) value String version Int @default(1) + isActive Boolean @default(true)
best practice	Use query parameters for optional filtering in getAllVariablesOfProject. For better API design and consistency, consider using query parameters instead of path parameters for optional filtering, sorting, and pagination options in getAllVariablesOfProject. apps/api/src/variable/controller/variable.controller.ts [92-102] -@Get('/all/:projectId') +@Get('/all') async getAllVariablesOfProject( @CurrentUser() user: User, - @Param('projectId') projectId: string, + @Query('projectId') projectId: string, @Query('page') page: number = 0, @Query('limit') limit: number = 10, @Query('sort') sort: string = 'name', @Query('order') order: string = 'asc', @Query('search') search: string = '' ) { Use uppercase for model fields for consistency and to avoid case sensitivity issues. For consistency and to avoid potential issues with case sensitivity in different environments, consider using uppercase for the Variable model fields id, name, createdAt, updatedAt, lastUpdatedById, projectId, and environmentId. apps/api/src/prisma/schema.prisma [353-357] model Variable { - id String @id @default(cuid()) - name String - versions VariableVersion[] // Stores the versions of the variable - createdAt DateTime @default(now()) - updatedAt DateTime @updatedAt + ID String @id @default(cuid()) + NAME String + VERSIONS VariableVersion[] // Stores the versions of the variable + CREATED_AT DateTime @default(now()) + UPDATED_AT DateTime @updatedAt Ensure the 'value' field in VariableVersion cannot be empty.                  To improve data integrity, consider adding constraints or checks to ensure that the value field in the VariableVersion model cannot be empty. apps/api/src/prisma/schema.prisma [339] -value String +value String @db.VarChar(255) @notEmpty Add deletion policies for Variable and VariableVersion models.                It's recommended to add a deletion policy for the Variable and VariableVersion models to handle cascading deletes or set nulls for related fields to maintain database integrity when deletions occur. apps/api/src/prisma/schema.prisma [352-355] model Variable { id String @id @default(cuid()) name String - versions VariableVersion[] // Stores the versions of the variable + versions VariableVersion[] @relation(onDelete: Cascade) // Stores the versions of the variable
maintainability	Organize variable event tests into separate describe blocks.                  To improve test isolation and readability, consider creating separate describe blocks for each controller action being tested, such as creating, fetching, and deleting variable events. apps/api/src/event/event.e2e.spec.ts [302] -it('should be able to fetch a variable event', async () => { - const newVariable = await variableService.createVariable( - user, - { - name: 'My variable', - value: 'My value', - environmentId: environment.id - }, - project.id - ) +describe('Variable Event Tests', () => { + it('should be able to create a variable event', async () => { + // Test for creating a variable event + }); + it('should be able to fetch a variable event', async () => { + // Test for fetching a variable event + }); + // Additional tests for variable events +}); Abstract repeated logic for checking variable existence into a separate method to reduce code duplication. To improve the readability and maintainability of your code, consider abstracting the logic for checking if a variable exists into a separate method. This logic is repeated in multiple places (createVariable, updateVariable, and updateVariableEnvironment) and can be refactored to reduce code duplication. apps/api/src/variable/service/variable.service.ts [69-72] -if (await this.variableExists(dto.name, environment.id)) { - throw new ConflictException( - `Variable already exists: ${dto.name} in environment ${environment.id} in project ${projectId}` - ) -} +await this.ensureVariableDoesNotExist(dto.name, environment.id, projectId);
security	Implement input validation for DTOs to enhance security and data integrity.   To ensure the security and integrity of your application, consider implementing input validation for all incoming DTOs. This is especially important for methods like createVariable and updateVariable where user input directly affects database operations. apps/api/src/variable/service/variable.service.ts [33-37] -async createVariable( - user: User, - dto: CreateVariable, - projectId: Project['id'] -) { +// Ensure dto and projectId are validated before proceeding with the method implementation
performance	Ensure pagination parameters are validated and sanitized for optimal performance and security. Consider implementing pagination in the getAllVariablesOfProject method to manage and optimize the performance of large datasets. This can be achieved by using the skip and take parameters dynamically based on the client's request. apps/api/src/variable/service/variable.service.ts [414-446] -return await this.prisma.variable.findMany({ - where: { - projectId, - name: { - contains: search - } - }, - include: { - versions: { - orderBy: { - version: 'desc' - }, - take: 1 - }, - lastUpdatedBy: { - select: { - id: true, - name: true - } - }, - environment: { - select: { - id: true, - name: true - } - } - }, - skip: page * limit, - take: limit, - orderBy: { - [sort]: order - } -}) +// The existing code already demonstrates a basic pagination implementation. Ensure that `page`, `limit`, `sort`, and `order` parameters are validated and sanitized.

---

✨ Improve tool usage guide:

---

Overview:
The improve tool scans the PR code changes, and automatically generates suggestions for improving the PR code. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on a PR.
When commenting, to edit configurations related to the improve tool (pr_code_suggestions section), use the following template:

/improve --pr_code_suggestions.some_config1=... --pr_code_suggestions.some_config2=...

With a configuration file, use the following template:

[pr_code_suggestions]
some_config1=...
some_config2=...

Enabling\disabling automation When you first install the app, the default mode for the improve tool is: pr_commands = ["/improve --pr_code_suggestions.summarize=true", ...] meaning the improve tool will run automatically on every PR, with summarization enabled. Delete this line to disable the tool from running automatically.

Utilizing extra instructions Extra instructions are very important for the improve tool, since they enable to guide the model to suggestions that are more relevant to the specific needs of the project. Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify relevant aspects that you want the model to focus on. Examples for extra instructions: [pr_code_suggestions] # /improve # extra_instructions=""" Emphasize the following aspects: - Does the code logic cover relevant edge cases? - Is the code logic clear and easy to understand? - Is the code logic efficient? ... """ Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

A note on code suggestions quality While the current AI for code is getting better and better (GPT-4), it's not flawless. Not all the suggestions will be perfect, and a user should not accept all of them automatically. Suggestions are not meant to be simplistic. Instead, they aim to give deep feedback and raise questions, ideas and thoughts to the user, who can then use his judgment, experience, and understanding of the code base. Recommended to use the 'extra_instructions' field to guide the model to suggestions that are more relevant to the specific needs of the project, or use the custom suggestions 💎 tool With large PRs, best quality will be obtained by using 'improve --extended' mode.

More PR-Agent commands To invoke the PR-Agent, add a comment using one of the following commands: /review: Request a review of your Pull Request. /describe: Update the PR title and description based on the contents of the PR. /improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback. /ask <QUESTION>: Ask a question about the PR. /update_changelog: Update the changelog based on the PR's contents. /add_docs 💎: Generate docstring for new components introduced in the PR. /generate_labels 💎: Generate labels for the PR based on the PR's contents. /analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component. See the tools guide for more details. To list the possible configuration parameters, add a /config comment.

See the improve usage page for a more comprehensive guide on using this tool.

[nx-cloud]

☁️ Nx Cloud Report

CI is running/has finished running commands for commit b61803a. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this CI Pipeline Execution

---

✅ Successfully ran 3 targets

• nx run api:test
• nx run api:prisma:generate
• nx run api:lint

---

Sent with 💌 from NxCloud.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (7bb3d21) 62.20% compared to head (b61803a) 89.24%.
Report is 14 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff              @@
##           develop     #149       +/-   ##
============================================
+ Coverage    62.20%   89.24%   +27.03%     
============================================
  Files           76       83        +7     
  Lines         1503     1637      +134     
  Branches       260      275       +15     
============================================
+ Hits           935     1461      +526     
+ Misses         568      176      -392     

Flag	Coverage Δ
api-e2e-tests	89.24% <100.00%> (+27.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[rajdip-b]

🎉 This PR is included in version 1.3.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀