Skip to content

Commit 24dd79b

Browse files
akhilmhdhakhilmhdh
committed
feat: added guides for new backend development
1 parent 5b9c043 commit 24dd79b

File tree

6 files changed

+195
-31
lines changed

6 files changed

+195
-31
lines changed

‎.env.migration.example

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
DB_CONNECTION_URI=

‎.gitignore

+1-1
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ node_modules
66
.env.gamma
77
.env.prod
88
.env.infisical
9-
9+
.env.migration
1010
*~
1111
*.swp
1212
*.swo

‎backend/docs/guide/create-feature-x.md

+105
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
# Guide on creating a feature in Infisical backend
2+
3+
Let's say you want to implement a new feature, call it feature-x for name sake. These are steps to follow.
4+
5+
## Database model change
6+
7+
If there is a database change, we must first address this to generate the database schemas for us to use.
8+
9+
| Create a `.env.migration` for setting the db connection uri for migration scripts or you can just export the env DB_CONNECTION_URI
10+
11+
1. if you have a new table, then go to `/src/db/schemas/models.ts` and update `TableName` enum to have the new table name.
12+
2. Then create a new migration file by running `npm run migration:new`, type the name. Keep it something related to what your about to do. For now `feature-x`
13+
3. Now go to `/src/db/migrations/<timestamp>_<feature-x>.ts`
14+
4. Here update both the function `up` and `down` to create/alter the postgres fields on migration up and to revert it back on migration down. [Keeping it idempotent](https://github.com/graphile/migrate/blob/main/docs/idempotent-examples.md).
15+
16+
### Generate TS schemas
17+
18+
Typically you would need to know write TS types for knex type sense. But we have automated this process
19+
20+
1. Start the server
21+
2. Run `npm run migration:latest` to apply all the changes to db
22+
3. Run `npm run generate:schema`. This will generate the type and schema using [zod](https://github.com/colinhacks/zod) in `/src/db/schemas` folder.
23+
4. Update the barrel export in `schema/index` and apply the new tables names in `/src/@types/knex.d.ts`. This will allow knex js to have typesense.
24+
25+
## Business Logic
26+
27+
With the database changes generated. Now let's create the APIs for `feature-x`.
28+
29+
1. Run `npm run generate:component`
30+
2. Select 1 that is service component
31+
3. Type service name in dashcase. Like `feature-x`
32+
33+
This will create a folder inside `/src/services` with `feature-x` and 3 files
34+
35+
1. `feature-x-dal`: The Database Access Layer function
36+
2. `feature-x-service`: The service layer where all bussiness logic happens
37+
3. `feature-x-type`: Types used by feature-x
38+
39+
There are more layers like for reusable shared function u can setup a file called `feature-x-fns`
40+
41+
You can use the custom infisical function `ormify` inside `src/lib/knex` to do simple db operations inside DAL.
42+
43+
## Connecting the service layer with server layer
44+
45+
All the server related logic happens inside `/src/server`. To connect the service layer inside server layer we use fastify plugins for dependency injection
46+
47+
1. Add the service type inside `fastify.d.ts` file below `service` namespace of a FastifyServerInstance type
48+
2. Now go to `/src/server/routes/index.ts`, instantiate the `feature-x` required dependencies like DAL layer and service layer and then pass it to `fastify.register("service,{...dependencies})`
49+
3. With this the service layer will be accessibile inside all routes under fastify service instance. It can be accessed with `server.services.<service name register>.<fn>`
50+
51+
## Writing the routes
52+
53+
1. To create a route component run `npm generate:component`
54+
2. Select option 3 by typing it out and then type the router name in dashcase.
55+
3. Provide the version number
56+
57+
This will generate a router file inside `src/server/routes/v<version-number>/<router component name>`
58+
59+
1. Add your logic to connect with service layer accordingly
60+
2. Then import the router component inside the version folder index.ts. Example, If the router component was inside v1, import the the function inside `v1/index.ts`
61+
3. Finally register it under proper prefix to access it.
62+
63+
The above contains the backend folder structure. All the contribution towards backend must follow the rules
64+
65+
- **scripts**: Contains all the reusable scripts used in backend automation like running migration, generating SQL schemas
66+
- **e2e-test**: The integration test for the APIs
67+
- **src**: Source code of backend
68+
69+
## Src
70+
71+
- **@types**: The type definition of some libraries like fastify, knex
72+
- **db**: Knexjs configuration required for database. Includes migration, seed files and sql type schemas
73+
- **lib**: Stateless reusable functions used throught code base
74+
- **queue**: Infisical queue system based on bullmq
75+
76+
### Server
77+
78+
- Anything related to fastify/service should be scoped inside here.
79+
- It contains the routes, fastify plugins, server configurations
80+
- Routes folder contains various version of routes seperated into v1,v2
81+
82+
### Services
83+
84+
- Core bussiness logic for all operations
85+
- Each service component follows co-location principle that is related things should be kept together
86+
- Each service component contains
87+
1. **dal**: The Database Access Layer function that contains all the db operations
88+
2. **service**: The service layer containing all the bussiness logic
89+
3. **type**: The type definition used inside the service component
90+
4. **fns**: Optional component to share reusable functions from a service related to another
91+
5. **queue**: Optional component to put queue specific logic for a component like `secret-queue.ts`
92+
93+
## EE
94+
95+
- Follows same pattern as above with an exception of licensn change from MIT -> Infisical Properitary License
96+
97+
### Notes
98+
99+
- All the services are interconnected at `/src/server/routes/index.ts`. We follow simple dependency injection principle
100+
- All files should be in dashcases.
101+
- Classes should not be used in codebase. Use simple functions to keep it simple
102+
- All commited code must be linted properly by running `npm run lint:fix` and type checked using `npm run type:check`
103+
- Try to avoid inter service shared logic as much as possible
104+
- A controller inside a router component should try to keep it calling only one service layer. This rule could have exception when another service
105+
like `audit-log` needs access to request object data. Then controller will call both the functions

‎backend/docs/guide/folder-structure.md

+81
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# Folder structure
2+
3+
```
4+
.
5+
├── scripts
6+
├── e2e-test
7+
└── src/
8+
├── @types/
9+
│ ├── knex.d.ts
10+
│ └── fastify.d.ts
11+
├── db/
12+
│ ├── migrations
13+
│ ├── schemas
14+
│ └── seed
15+
├── lib/
16+
│ ├── fn
17+
│ ├── date
18+
│ └── config
19+
├── queue
20+
├── server/
21+
│ ├── routes/
22+
│ │ ├── v1
23+
│ │ └── v2
24+
│ ├── plugins
25+
│ └── config
26+
├── services/
27+
│ ├── auth
28+
│ ├── org
29+
│ └── project/
30+
│ ├── project-service.ts
31+
│ ├── project-types.ts
32+
│ └── project-dal.ts
33+
└── ee/
34+
├── routes
35+
└── services
36+
```
37+
38+
The above contains the backend folder structure. All the contribution towards backend must follow the rules
39+
40+
- **scripts**: Contains all the reusable scripts used in backend automation like running migration, generating SQL schemas
41+
- **e2e-test**: The integration test for the APIs
42+
- **src**: Source code of backend
43+
44+
## SRC
45+
46+
- **@types**: The type definition of some libraries like fastify, knex
47+
- **db**: Knexjs configuration required for database. Includes migration, seed files and sql type schemas
48+
- **lib**: Stateless reusable functions used throught code base
49+
- **queue**: Infisical queue system based on bullmq
50+
51+
### Server
52+
53+
- Anything related to fastify/service should be scoped inside here.
54+
- It contains the routes, fastify plugins, server configurations
55+
- Routes folder contains various version of routes seperated into v1,v2
56+
57+
### Services
58+
59+
- Core bussiness logic for all operations
60+
- Each service component follows co-location principle that is related things should be kept together
61+
- Each service component contains
62+
63+
1. **dal**: The Database Access Layer function that contains all the db operations
64+
2. **service**: The service layer containing all the bussiness logic
65+
3. **type**: The type definition used inside the service component
66+
4. **fns**: Optional component to share reusable functions from a service related to another
67+
5. **queue**: Optional component to put queue specific logic for a component like `secret-queue.ts`
68+
69+
## EE
70+
71+
- Follows same pattern as above with an exception of licensn change from MIT -> Infisical Properitary License
72+
73+
### Notes
74+
75+
- All the services are interconnected at `/src/server/routes/index.ts`. We follow simple dependency injection principle
76+
- All files should be in dashcases.
77+
- Classes should not be used in codebase. Use simple functions to keep it simple
78+
- All commited code must be linted properly by running `npm run lint:fix` and type checked using `npm run type:check`
79+
- Try to avoid inter service shared logic as much as possible
80+
- A controller inside a router component should try to keep it calling only one service layer. This rule could have exception when another service
81+
like `audit-log` needs access to request object data. Then controller will call both the functions

‎backend/scripts/generate-schema-types.ts

+6-29
Original file line numberDiff line numberDiff line change
@@ -3,13 +3,9 @@ import dotenv from "dotenv";
33
import path from "path";
44
import knex from "knex";
55
import { writeFileSync } from "fs";
6-
import promptSync from "prompt-sync";
7-
8-
const prompt = promptSync({ sigint: true });
96

107
dotenv.config({
11-
path: path.join(__dirname, "../.env"),
12-
debug: true
8+
path: path.join(__dirname, "../../.env.migration")
139
});
1410

1511
const db = knex({
@@ -94,17 +90,7 @@ const main = async () => {
9490
.orderBy("table_name")
9591
).filter((el) => !el.tableName.includes("_migrations"));
9692

97-
console.log("Select a table to generate schema");
98-
console.table(tables);
99-
console.log("all: all tables");
100-
const selectedTables = prompt("Type table numbers comma seperated: ");
101-
const tableNumbers =
102-
selectedTables !== "all" ? selectedTables.split(",").map((el) => Number(el)) : [];
103-
10493
for (let i = 0; i < tables.length; i += 1) {
105-
// skip if not desired table
106-
if (selectedTables !== "all" && !tableNumbers.includes(i)) continue;
107-
10894
const { tableName } = tables[i];
10995
const columns = await db(tableName).columnInfo();
11096
const columnNames = Object.keys(columns);
@@ -124,16 +110,16 @@ const main = async () => {
124110
if (colInfo.nullable) {
125111
ztype = ztype.concat(".nullable().optional()");
126112
}
127-
schema = schema.concat(`${!schema ? "\n" : ""} ${columnName}: ${ztype},\n`);
113+
schema = schema.concat(
114+
`${!schema ? "\n" : ""} ${columnName}: ${ztype}${colNum === columnNames.length - 1 ? "" : ","}\n`
115+
);
128116
}
129117

130118
const dashcase = tableName.split("_").join("-");
131119
const pascalCase = tableName
132120
.split("_")
133-
.reduce(
134-
(prev, curr) => prev + `${curr.at(0)?.toUpperCase()}${curr.slice(1).toLowerCase()}`,
135-
""
136-
);
121+
.reduce((prev, curr) => prev + `${curr.at(0)?.toUpperCase()}${curr.slice(1).toLowerCase()}`, "");
122+
137123
writeFileSync(
138124
path.join(__dirname, "../src/db/schemas", `${dashcase}.ts`),
139125
`// Code generated by automation script, DO NOT EDIT.
@@ -152,15 +138,6 @@ export type T${pascalCase}Insert = Omit<T${pascalCase}, TImmutableDBKeys>;
152138
export type T${pascalCase}Update = Partial<Omit<T${pascalCase}, TImmutableDBKeys>>;
153139
`
154140
);
155-
156-
// const file = readFileSync(path.join(__dirname, "../src/db/schemas/index.ts"), "utf8");
157-
// if (!file.includes(`export * from "./${dashcase};"`)) {
158-
// appendFileSync(
159-
// path.join(__dirname, "../src/db/schemas/index.ts"),
160-
// `\nexport * from "./${dashcase}";`,
161-
// "utf8"
162-
// );
163-
// }
164141
}
165142

166143
process.exit(0);

‎backend/src/db/knexfile.ts

+1-1
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ import path from "path";
77

88
// Update with your config settings.
99
dotenv.config({
10-
path: path.join(__dirname, "../../.env"),
10+
path: path.join(__dirname, "../../../.env.migration"),
1111
debug: true
1212
});
1313
export default {

0 commit comments

Comments
 (0)