Skip to content

Commit 59ede4c

Browse files
victor-yanevebadiere
victor-yanev
and
ebadiere
authoredJul 30, 2024··
chore: Implement memory leak detection in tests (#2695)
* chore: Implement memory leak detection in tests Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: Implement memory leak detection in tests Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: Implement memory leak detection in tests Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: Implement memory leak detection in tests Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: Implement memory leak detection in tests Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: add docs Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: fix sonar issues Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: formatting Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: small fix Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: trace GC and write snapshot if memory leak > 0.5 MB Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: revert false changes to eth.ts Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: optimize code Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * Merge branch 'main' into 2260-Implement-a-Memory-Leak-detection-Test Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> # Conflicts: # package-lock.json # package.json # packages/server/tsconfig.json * fix: Do not write heap snapshots for acceptance tests Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * fix: add TODO for removing --trace_gc flag Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * test: add github comment in cases a test is having memory leaks Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * test: add github context env variables Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * test: extract common logic for github api calls Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * test: console.debug -> console.log for successful PR comments Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * test: fix request to github API Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * Merge branch 'main' into 2260-Implement-a-Memory-Leak-detection-Test Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> # Conflicts: # packages/server/tests/acceptance/index.spec.ts * Merge branch 'main' into 2260-Implement-a-Memory-Leak-detection-Test Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> # Conflicts: # packages/server/tests/acceptance/index.spec.ts * fix: request to github API Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: add upload heap snapshots step in github workflow for acceptance and integration tests Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * fix: add `subject_type: file` to github request for adding comment on PR Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * fix: fix path in request to github API Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: extract github request logic in new file - githubClient.ts Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * fix: path of upload-artifact action for uploading heap snapshots Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * fix: requests in githubClient.ts Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * fix: requests in githubClient.ts Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * fix: format message of github-actions[bot] Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * fix: format message of github-actions[bot] Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: remove unused github context env variable Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: formatting Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: formatting Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: formatting Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * fix: `formatBytes` returns `NaN undefined` when `0` is passed to it Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * fix: `formatBytes` returns `NaN undefined` when negative number is passed to it Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * fix: predicate for updating existing comment Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * fix: remove unused constant + temporary console logs Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * fix: make memory leak report more descriptive Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: remove unnecessary console logs Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: reduce MEMORY_LEAK_SNAPSHOT_THRESHOLD to 500 KB Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * fix: update logic for detecting memory leak based on heap size threshold Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * fix: add separate threshold for taking heap snapshots Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * fix: set `WRITE_SNAPSHOT_ON_MEMORY_LEAK` to be `false` by default Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * test: add warm-up phase to memory leak detection Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> * chore: revert changes to rpc_batch1.spec.ts Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> --------- Signed-off-by: Victor Yanev <victor.yanev@limechain.tech> Signed-off-by: Eric Badiere <ebadiere@gmail.com> Co-authored-by: Eric Badiere <ebadiere@gmail.com>
·
v0.67.0-rc1v0.53.0-rc1
1 parent 2b06cb7 commit 59ede4c

20 files changed

+693
-148
lines changed
 

‎.github/workflows/acceptance-workflow.yml

+11
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,17 @@ jobs:
9090
env:
9191
TEST_WS_SERVER: ${{ inputs.test_ws_server }}
9292
SUBSCRIPTIONS_ENABLED: ${{ inputs.test_ws_server }}
93+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
94+
GITHUB_PR_NUMBER: ${{ github.event.number }}
95+
GITHUB_REPOSITORY: ${{ github.repository }}
96+
97+
- name: Upload Heap Snapshots
98+
if: ${{ !cancelled() }}
99+
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
100+
with:
101+
name: Heap Snapshots
102+
path: "**/*.heapsnapshot"
103+
if-no-files-found: ignore
93104

94105
- name: Upload Test Results
95106
if: always()

‎.github/workflows/test.yml

+12
Original file line numberDiff line numberDiff line change
@@ -42,8 +42,20 @@ jobs:
4242
run: npm install -g pnpm
4343

4444
- name: Build Typescript and Run tests
45+
env:
46+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
47+
GITHUB_PR_NUMBER: ${{ github.event.number }}
48+
GITHUB_REPOSITORY: ${{ github.repository }}
4549
run: npm run build-and-test
4650

51+
- name: Upload Heap Snapshots
52+
if: ${{ !cancelled() }}
53+
uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
54+
with:
55+
name: Heap Snapshots
56+
path: "**/*.heapsnapshot"
57+
if-no-files-found: ignore
58+
4759
- name: Upload coverage report
4860
if: ${{ always() && !cancelled() }}
4961
run: node_modules/codecov/bin/codecov

‎package-lock.json

+57-4
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎package.json

+1
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
"devDependencies": {
44
"@types/chai-as-promised": "^7.1.5",
55
"@types/co-body": "6.1.0",
6+
"@types/koa-cors": "^0.0.6",
67
"@typescript-eslint/eslint-plugin": "^6.5.0",
78
"@typescript-eslint/parser": "^6.5.0",
89
"axios-mock-adapter": "^1.20.0",

‎packages/relay/src/lib/clients/cache/redisCache.ts

+5-2
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,7 @@ export class RedisCache implements ICacheClient {
118118
* @param {string} [requestIdPrefix] - The optional request ID prefix.
119119
* @returns {Promise<any | null>} The cached value or null if not found.
120120
*/
121-
async get(key: string, callingMethod: string, requestIdPrefix?: string | undefined) {
121+
async get(key: string, callingMethod: string, requestIdPrefix?: string | undefined): Promise<any | null> {
122122
const client = await this.getConnectedClient();
123123
const result = await client.get(key);
124124
if (result) {
@@ -244,6 +244,9 @@ export class RedisCache implements ICacheClient {
244244
}
245245

246246
async disconnect(): Promise<void> {
247-
await (await this.getConnectedClient()).disconnect();
247+
await this.getConnectedClient().then((client) => {
248+
client.disconnect();
249+
client.unsubscribe();
250+
});
248251
}
249252
}

‎packages/relay/tests/test.env

+2
Original file line numberDiff line numberDiff line change
@@ -26,3 +26,5 @@ HAPI_CLIENT_ERROR_RESET= [50]
2626
FILTER_API_ENABLED=true
2727
DEBUG_API_ENABLED=true
2828
TEST=true
29+
MEMWATCH_ENABLED=true
30+
WRITE_SNAPSHOT_ON_MEMORY_LEAK=false

‎packages/server/package.json

+2
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
"axios": "^1.4.0",
1111
"co-body": "6.2.0",
1212
"dotenv": "^16.0.0",
13+
"heapdump": "^0.3.15",
1314
"koa": "^2.13.4",
1415
"koa-body-parser": "^0.2.1",
1516
"koa-cors": "^0.0.16",
@@ -28,6 +29,7 @@
2829
"@types/chai": "^4.3.0",
2930
"@types/cors": "^2.8.12",
3031
"@types/express": "^4.17.13",
32+
"@types/heapdump": "^0.3.4",
3133
"@types/koa-bodyparser": "^4.3.5",
3234
"@types/koa-router": "^7.4.4",
3335
"@types/mocha": "^9.1.0",

‎packages/server/src/server.ts

+1-1
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ import path from 'path';
2727
import fs from 'fs';
2828
import { v4 as uuid } from 'uuid';
2929
import { formatRequestIdMessage } from './formatters';
30+
import cors from 'koa-cors';
3031

3132
const mainLogger = pino({
3233
name: 'hedera-json-rpc-relay',
@@ -40,7 +41,6 @@ const mainLogger = pino({
4041
},
4142
});
4243

43-
const cors = require('koa-cors');
4444
const logger = mainLogger.child({ name: 'rpc-server' });
4545
const register = new Registry();
4646
const relay: Relay = new RelayImpl(logger.child({ name: 'relay' }), register);

‎packages/server/tests/acceptance/index.spec.ts

+6
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ import dotenv from 'dotenv';
2323
import path from 'path';
2424
import pino from 'pino';
2525
import chaiAsPromised from 'chai-as-promised';
26+
import { GCProfiler } from 'v8';
2627

2728
// Other external resources
2829
import fs from 'fs';
@@ -103,6 +104,11 @@ describe('RPC Server Acceptance Tests', function () {
103104
}
104105
};
105106

107+
// leak detection middleware
108+
if (process.env.MEMWATCH_ENABLED === 'true') {
109+
Utils.captureMemoryLeaks(new GCProfiler());
110+
}
111+
106112
before(async () => {
107113
// configuration details
108114
logger.info('Acceptance Tests Configurations successfully loaded');

‎packages/server/tests/clients/githubClient.ts

+140
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
/*-
2+
*
3+
* Hedera JSON RPC Relay
4+
*
5+
* Copyright (C) 2024 Hedera Hashgraph, LLC
6+
*
7+
* Licensed under the Apache License, Version 2.0 (the "License");
8+
* you may not use this file except in compliance with the License.
9+
* You may obtain a copy of the License at
10+
*
11+
* http://www.apache.org/licenses/LICENSE-2.0
12+
*
13+
* Unless required by applicable law or agreed to in writing, software
14+
* distributed under the License is distributed on an "AS IS" BASIS,
15+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16+
* See the License for the specific language governing permissions and
17+
* limitations under the License.
18+
*
19+
*/
20+
21+
import { Octokit } from '@octokit/core';
22+
import { GitHubContext } from '../types/GitHubContext';
23+
24+
/**
25+
* Client for interacting with GitHub, providing methods to perform operations such as adding comments to pull requests.
26+
*/
27+
export class GitHubClient {
28+
private static readonly GET_COMMENTS_ENDPOINT = 'GET /repos/{owner}/{repo}/issues/{issue_number}/comments';
29+
private static readonly CREATE_COMMENT_ENDPOINT = 'POST /repos/{owner}/{repo}/issues/{issue_number}/comments';
30+
private static readonly UPDATE_COMMENT_ENDPOINT = 'PATCH /repos/{owner}/{repo}/issues/comments/{comment_id}';
31+
32+
/**
33+
* The Octokit instance used to interact with GitHub.
34+
* @private
35+
*/
36+
private readonly octokit: Octokit;
37+
38+
constructor(octokit?: Octokit) {
39+
this.octokit = octokit || new Octokit({ auth: process.env.GITHUB_TOKEN });
40+
}
41+
42+
/**
43+
* Update or add a comment to a pull request.
44+
* @param {string} commentBody - The body of the comment.
45+
* @param {function} predicate - A function that determines if an existing comment should be updated.
46+
* @returns {Promise<void>} A promise that resolves when the comment is successfully updated or added.
47+
*/
48+
async addOrUpdateExistingCommentOnPullRequest(
49+
commentBody: string,
50+
predicate: (existingComment: string) => boolean,
51+
): Promise<void> {
52+
const comments = await this.getCommentsOnPullRequest();
53+
const existingComment = comments.data.find((comment) => comment.body && predicate(comment.body));
54+
if (existingComment) {
55+
await this.updateCommentOnPullRequest(commentBody, existingComment.id);
56+
} else {
57+
await this.addCommentToPullRequest(commentBody);
58+
}
59+
}
60+
61+
/**
62+
* Gets a list of comments on a pull request.
63+
* @returns A promise that resolves with the list of comments.
64+
*/
65+
async getCommentsOnPullRequest() {
66+
try {
67+
const context = GitHubClient.getContext();
68+
return await this.octokit.request(GitHubClient.GET_COMMENTS_ENDPOINT, {
69+
owner: context.owner,
70+
repo: context.repo,
71+
issue_number: context.pullNumber,
72+
});
73+
} catch (error) {
74+
console.error('Failed to retrieve comments on PR:', error);
75+
return { data: [] };
76+
}
77+
}
78+
79+
/**
80+
* Updates a comment on a pull request.
81+
* @param {string} commentBody - The body of the comment.
82+
* @param {number} commentId - The ID of the comment to update.
83+
* @returns {Promise<void>} A promise that resolves when the comment is successfully updated.
84+
*/
85+
async updateCommentOnPullRequest(commentBody: string, commentId: number): Promise<void> {
86+
try {
87+
const context = GitHubClient.getContext();
88+
await this.octokit.request(GitHubClient.UPDATE_COMMENT_ENDPOINT, {
89+
owner: context.owner,
90+
repo: context.repo,
91+
comment_id: commentId,
92+
body: commentBody,
93+
});
94+
} catch (error) {
95+
console.error('Failed to update comment on PR:', error);
96+
}
97+
}
98+
99+
/**
100+
* Adds a comment to a pull request.
101+
* @param {string} commentBody - The body of the comment.
102+
* @returns {Promise<void>} A promise that resolves when the comment is successfully posted.
103+
*/
104+
async addCommentToPullRequest(commentBody: string): Promise<void> {
105+
try {
106+
const context = GitHubClient.getContext();
107+
await this.octokit.request(GitHubClient.CREATE_COMMENT_ENDPOINT, {
108+
owner: context.owner,
109+
repo: context.repo,
110+
issue_number: context.pullNumber,
111+
body: commentBody,
112+
});
113+
} catch (error) {
114+
console.error('Failed to post comment to PR:', error);
115+
}
116+
}
117+
118+
/**
119+
* Retrieves the GitHub context from environment variables.
120+
* @returns {GitHubContext} The GitHub context.
121+
*/
122+
private static getContext(): GitHubContext {
123+
const { GITHUB_REPOSITORY, GITHUB_PR_NUMBER, GITHUB_TOKEN } = process.env;
124+
if (!GITHUB_REPOSITORY || !GITHUB_PR_NUMBER || !GITHUB_TOKEN) {
125+
throw new Error(`Missing required environment variables: $GITHUB_REPOSITORY, $GITHUB_PR_NUMBER, $GITHUB_TOKEN`);
126+
}
127+
128+
const pullNumber = parseInt(GITHUB_PR_NUMBER);
129+
if (isNaN(pullNumber)) {
130+
throw new Error('Invalid PR number: $GITHUB_PR_NUMBER must be a valid number.');
131+
}
132+
133+
const [owner, repo] = GITHUB_REPOSITORY.split('/');
134+
if (!owner || !repo) {
135+
throw new Error('Invalid $GITHUB_REPOSITORY format: Expected "owner/repo".');
136+
}
137+
138+
return { owner, repo, pullNumber, token: GITHUB_TOKEN };
139+
}
140+
}

‎packages/server/tests/helpers/utils.ts

+239-3
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,19 @@ import { AccountId, KeyList, PrivateKey } from '@hashgraph/sdk';
2828
import { AliasAccount } from '../types/AliasAccount';
2929
import ServicesClient from '../clients/servicesClient';
3030
import http from 'http';
31+
import { GCProfiler, setFlagsFromString } from 'v8';
32+
import { runInNewContext } from 'vm';
33+
import { Context } from 'mocha';
34+
import { writeSnapshot } from 'heapdump';
35+
import { GitHubClient } from '../clients/githubClient';
36+
import MirrorClient from '../clients/mirrorClient';
37+
import { HeapDifferenceStatistics } from '../types/HeapDifferenceStatistics';
3138

3239
export class Utils {
40+
static readonly HEAP_SIZE_DIFF_MEMORY_LEAK_THRESHOLD: number = 5e5; // 500 KB
41+
static readonly HEAP_SIZE_DIFF_SNAPSHOT_THRESHOLD: number = 1e6; // 1 MB
42+
static readonly WARM_UP_TEST_COUNT: number = 3;
43+
3344
/**
3445
* Converts a number to its hexadecimal representation.
3546
*
@@ -248,11 +259,11 @@ export class Utils {
248259
* @param {MirrorClient} mirrorNode The mirror node client.
249260
* @param {AliasAccount} creator The creator account for the alias.
250261
* @param {string} requestId The unique identifier for the request.
251-
* @param {string} balanceInWeiBars The initial balance for the alias account in wei bars. Defaults to 10 HBAR (10,000,000,000,000,000,000 wei).
262+
* @param {string} balanceInTinyBar The initial balance for the alias account in tiny bars. Defaults to 10 HBAR.
252263
* @returns {Promise<AliasAccount>} A promise resolving to the created alias account.
253264
*/
254265
static readonly createAliasAccount = async (
255-
mirrorNode,
266+
mirrorNode: MirrorClient,
256267
creator: AliasAccount,
257268
requestId: string,
258269
balanceInTinyBar: string = '1000000000', //10 HBAR
@@ -292,7 +303,7 @@ export class Utils {
292303
};
293304

294305
static async createMultipleAliasAccounts(
295-
mirrorNode,
306+
mirrorNode: MirrorClient,
296307
initialAccount: AliasAccount,
297308
neededAccounts: number,
298309
initialAmountInTinyBar: string,
@@ -372,4 +383,229 @@ export class Utils {
372383
static async wait(time: number): Promise<void> {
373384
await new Promise((r) => setTimeout(r, time));
374385
}
386+
387+
static async writeHeapSnapshotAsync(): Promise<string | undefined> {
388+
return new Promise((resolve, reject) => {
389+
writeSnapshot((error, fileName) => {
390+
if (error) {
391+
reject(error);
392+
}
393+
console.info(`Heap snapshot written to ${fileName}`);
394+
resolve(fileName);
395+
});
396+
});
397+
}
398+
399+
/**
400+
* Captures memory leaks in the test suite.
401+
* The function will start the profiler before each test and stop it after each test.
402+
* If a memory leak is detected, the function will log the difference in memory usage.
403+
*/
404+
static captureMemoryLeaks(profiler: GCProfiler): void {
405+
setFlagsFromString('--expose_gc');
406+
const gc = runInNewContext('gc');
407+
const githubClient = new GitHubClient();
408+
409+
let isWarmUpCompleted = false;
410+
411+
const warmUp = async () => {
412+
for (let i = 0; i < Utils.WARM_UP_TEST_COUNT; i++) {
413+
// Run dummy tests to warm up the environment
414+
await new Promise((resolve) => setTimeout(resolve, 100));
415+
}
416+
isWarmUpCompleted = true;
417+
};
418+
419+
beforeEach(async function () {
420+
if (!isWarmUpCompleted) {
421+
await warmUp();
422+
}
423+
profiler.start();
424+
});
425+
426+
afterEach(async function (this: Context) {
427+
this.timeout(60000);
428+
await gc(); // force a garbage collection to get accurate memory usage
429+
try {
430+
const result = profiler.stop();
431+
const statsGrowingHeapSize = result.statistics.filter((stats) => {
432+
return stats.afterGC.heapStatistics.totalHeapSize > stats.beforeGC.heapStatistics.totalHeapSize;
433+
});
434+
const totalDiffBytes = statsGrowingHeapSize.reduce((acc, stats) => {
435+
const diff = stats.afterGC.heapStatistics.totalHeapSize - stats.beforeGC.heapStatistics.totalHeapSize;
436+
return acc + diff;
437+
}, 0);
438+
const isPotentialMemoryLeak = totalDiffBytes > Utils.HEAP_SIZE_DIFF_MEMORY_LEAK_THRESHOLD;
439+
440+
if (isPotentialMemoryLeak) {
441+
console.warn('Potential memory leak detected!');
442+
const statsDiff: HeapDifferenceStatistics = statsGrowingHeapSize.map((stats) => ({
443+
gcType: stats.gcType,
444+
cost: stats.cost,
445+
diffGC: {
446+
heapStatistics: Utils.difference(stats.afterGC.heapStatistics, stats.beforeGC.heapStatistics),
447+
heapSpaceStatistics: Utils.difference(
448+
stats.afterGC.heapSpaceStatistics,
449+
stats.beforeGC.heapSpaceStatistics,
450+
).filter((spaceStatistics) => spaceStatistics.spaceSize > 0),
451+
},
452+
}));
453+
console.error(
454+
`Total Heap Size ${Utils.formatBytes(totalDiffBytes)}: --> ` + JSON.stringify(statsDiff, null, 2),
455+
);
456+
// add comment on PR highlighting after which test the memory leak is happening
457+
const testTitle = this.currentTest?.title ?? 'Unknown test';
458+
const comment = Utils.generateMemoryLeakComment(testTitle, statsDiff);
459+
await githubClient.addOrUpdateExistingCommentOnPullRequest(comment, (existing: string) =>
460+
existing.includes(`\`${testTitle}\``),
461+
);
462+
// write a heap snapshot if the memory leak is more than 1 MB
463+
const isMemoryLeakSnapshotEnabled = process.env.WRITE_SNAPSHOT_ON_MEMORY_LEAK === 'true';
464+
if (isMemoryLeakSnapshotEnabled && totalDiffBytes > Utils.HEAP_SIZE_DIFF_SNAPSHOT_THRESHOLD) {
465+
console.info('Writing heap snapshot...');
466+
await Utils.writeHeapSnapshotAsync();
467+
}
468+
}
469+
} catch (error) {
470+
console.error('Error capturing memory leaks:', error);
471+
}
472+
});
473+
}
474+
475+
/**
476+
* Generates a comment indicating a memory leak detected during tests.
477+
* @param {string} testTitle The title of the current test.
478+
* @param {HeapDifferenceStatistics} statsDiff The difference in memory statistics indicating the leak.
479+
* @returns {string} The formatted comment.
480+
*/
481+
private static generateMemoryLeakComment(testTitle: string, statsDiff: HeapDifferenceStatistics): string {
482+
const commentHeader = '## 🚨 Memory Leak Detected 🚨';
483+
const summary = `A potential memory leak has been detected in the test titled \`${testTitle}\`. This may impact the application's performance and stability.`;
484+
const detailsHeader = '### Details';
485+
const formattedStatsDiff = this.formatHeapDifferenceStatistics(statsDiff);
486+
const recommendationsHeader = '### Recommendations';
487+
const recommendations =
488+
'Please investigate the memory allocations in this test, focusing on objects that are not being properly deallocated.';
489+
490+
return `${commentHeader}\n\n${summary}\n\n${detailsHeader}\n${formattedStatsDiff}\n\n${recommendationsHeader}\n${recommendations}`;
491+
}
492+
493+
/**
494+
* Formats the difference in heap statistics into a readable string.
495+
* @param {HeapDifferenceStatistics} statsDiff The difference in heap statistics.
496+
* @returns {string} The formatted string.
497+
*/
498+
private static formatHeapDifferenceStatistics(statsDiff: HeapDifferenceStatistics): string {
499+
let message = '📊 **Memory Leak Detection Report** 📊\n\n';
500+
501+
statsDiff.forEach((entry) => {
502+
message += `**GC Type**: ${entry.gcType}\n`;
503+
message += `**Cost**: ${entry.cost.toLocaleString()} ms\n\n`;
504+
message += '**Heap Statistics (before vs after executing the test)**:\n';
505+
Object.entries(entry.diffGC.heapStatistics).forEach(([key, value]) => {
506+
message += `- **${this.camelCaseToTitleCase(key)}**: ${this.formatBytes(value)}\n`;
507+
});
508+
message += '\n**Heap Space Statistics (before vs after executing the test)**:\n';
509+
entry.diffGC.heapSpaceStatistics.forEach((space) => {
510+
message += ` - **${this.snakeCaseToTitleCase(space.spaceName)}**:\n`;
511+
Object.entries(space).forEach(([key, value]) => {
512+
if (key !== 'spaceName') {
513+
message += ` - **${this.camelCaseToTitleCase(key)}**: ${this.formatBytes(value)}\n`;
514+
}
515+
});
516+
message += '\n';
517+
});
518+
});
519+
520+
return message;
521+
}
522+
523+
/**
524+
* Converts a string in camel case to title case.
525+
* @param textInCamelCase The text in camel case.
526+
* @return The text in title case.
527+
*/
528+
private static camelCaseToTitleCase(textInCamelCase: string): string {
529+
return textInCamelCase
530+
.replace(/([A-Z])/g, ' $1')
531+
.replace(/^./, (str) => str.toUpperCase())
532+
.trim();
533+
}
534+
535+
/**
536+
* Converts a string in snake case to title case.
537+
* @param textInSnakeCase The text in snake case.
538+
* @return The text in title case.
539+
*/
540+
private static snakeCaseToTitleCase(textInSnakeCase: string): string {
541+
return textInSnakeCase
542+
.split('_')
543+
.map((word) => word.charAt(0).toUpperCase() + word.slice(1).toLowerCase())
544+
.join(' ')
545+
.trim();
546+
}
547+
548+
/**
549+
* Calculates the difference between two objects or arrays of objects.
550+
* This utility method is used to calculate the difference in heap statistics before and after GC.
551+
* @param after The object representing the state after an operation.
552+
* @param before The object representing the state before the operation.
553+
* @returns The difference between the two states.
554+
*/
555+
private static difference<T extends number | string | object | object[]>(after: T, before: T): T {
556+
if (Array.isArray(after) && Array.isArray(before)) {
557+
return this.arrayDifference(after, before);
558+
} else if (typeof after === 'object' && typeof before === 'object') {
559+
return this.objectDifference(after, before);
560+
} else if (typeof after === 'number' && typeof before === 'number') {
561+
return (after - before) as T;
562+
} else if (typeof after === 'string' && typeof before === 'string') {
563+
if (after !== before) {
564+
throw new Error(`Mismatched values: ${after} is not equal to ${before}`);
565+
}
566+
return after as T;
567+
} else {
568+
throw new Error('Invalid input: both parameters must be objects or arrays of objects');
569+
}
570+
}
571+
572+
/**
573+
* Calculates the difference between two objects
574+
* @param after
575+
* @param before
576+
*/
577+
private static objectDifference<T extends object>(after: T, before: T): T {
578+
const diff = { ...after };
579+
for (const key of Object.keys(after)) {
580+
if (!(key in before)) {
581+
throw new Error(`Mismatched properties: ${key} is not present in both objects`);
582+
}
583+
diff[key] = this.difference(after[key], before[key]);
584+
}
585+
return diff as T;
586+
}
587+
588+
/**
589+
* Calculates the difference between two arrays of objects
590+
* @param after
591+
* @param before
592+
*/
593+
private static arrayDifference<T extends object[]>(after: T, before: T): T {
594+
return after.map((item: object, index: number) => this.difference(item, before[index])) as T;
595+
}
596+
597+
/**
598+
* Formats bytes into a readable string.
599+
* @param {number} bytes The number of bytes.
600+
* @returns {string} A formatted string representing the size in bytes, KB, MB, GB, or TB.
601+
*/
602+
private static formatBytes(bytes: number): string {
603+
if (bytes === 0) return 'no changes';
604+
const prefix = bytes > 0 ? 'increased with' : 'decreased with';
605+
const units = ['bytes', 'KB', 'MB', 'GB', 'TB'];
606+
let power = Math.floor(Math.log(Math.abs(bytes)) / Math.log(1000));
607+
power = Math.min(power, units.length - 1);
608+
const size = Math.abs(bytes) / Math.pow(1000, power);
609+
return `${prefix} ${size.toFixed(2)} ${units[power]}`;
610+
}
375611
}

‎packages/server/tests/integration/server.spec.ts

+151-136
Large diffs are not rendered by default.

‎packages/server/tests/integration/test.env

+1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
E2E_SERVER_PORT=7546
22
CHAIN_ID=0x12a
3+
TEST=true

‎packages/server/tests/localAcceptance.env

+2
Original file line numberDiff line numberDiff line change
@@ -26,3 +26,5 @@ WS_NEW_HEADS_ENABLED=false
2626
INITIAL_BALANCE='5000000000'
2727
LIMIT_DURATION=90000
2828
SERVER_REQUEST_TIMEOUT_MS=60000
29+
MEMWATCH_ENABLED=true
30+
WRITE_SNAPSHOT_ON_MEMORY_LEAK=false

‎packages/server/tests/previewnetAcceptance.env

+2-1
Original file line numberDiff line numberDiff line change
@@ -17,4 +17,5 @@ DEBUG_API_ENABLED=true
1717
TEST_GAS_PRICE_DEVIATION=0.2
1818
WS_NEW_HEADS_ENABLED=true
1919
SERVER_REQUEST_TIMEOUT_MS=60000
20-
20+
MEMWATCH_ENABLED=true
21+
WRITE_SNAPSHOT_ON_MEMORY_LEAK=false

‎packages/server/tests/testnetAcceptance.env

+2
Original file line numberDiff line numberDiff line change
@@ -16,3 +16,5 @@ FILTER_API_ENABLED=true
1616
DEBUG_API_ENABLED=true
1717
TEST_GAS_PRICE_DEVIATION=0.2
1818
SERVER_REQUEST_TIMEOUT_MS=60000
19+
MEMWATCH_ENABLED=true
20+
WRITE_SNAPSHOT_ON_MEMORY_LEAK=false

‎packages/server/tests/types/GitHubContext.ts

+26
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
/*-
2+
*
3+
* Hedera JSON RPC Relay
4+
*
5+
* Copyright (C) 2024 Hedera Hashgraph, LLC
6+
*
7+
* Licensed under the Apache License, Version 2.0 (the "License");
8+
* you may not use this file except in compliance with the License.
9+
* You may obtain a copy of the License at
10+
*
11+
* http://www.apache.org/licenses/LICENSE-2.0
12+
*
13+
* Unless required by applicable law or agreed to in writing, software
14+
* distributed under the License is distributed on an "AS IS" BASIS,
15+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16+
* See the License for the specific language governing permissions and
17+
* limitations under the License.
18+
*
19+
*/
20+
21+
export interface GitHubContext {
22+
readonly owner: string;
23+
readonly repo: string;
24+
readonly token: string;
25+
readonly pullNumber: number;
26+
}

‎packages/server/tests/types/HeapDifferenceStatistics.ts

+30
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
/*-
2+
*
3+
* Hedera JSON RPC Relay
4+
*
5+
* Copyright (C) 2024 Hedera Hashgraph, LLC
6+
*
7+
* Licensed under the Apache License, Version 2.0 (the "License");
8+
* you may not use this file except in compliance with the License.
9+
* You may obtain a copy of the License at
10+
*
11+
* http://www.apache.org/licenses/LICENSE-2.0
12+
*
13+
* Unless required by applicable law or agreed to in writing, software
14+
* distributed under the License is distributed on an "AS IS" BASIS,
15+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
16+
* See the License for the specific language governing permissions and
17+
* limitations under the License.
18+
*
19+
*/
20+
21+
import { HeapSpaceStatistics, HeapStatistics } from 'v8';
22+
23+
export type HeapDifferenceStatistics = Array<{
24+
gcType: string;
25+
cost: number;
26+
diffGC: {
27+
heapStatistics: HeapStatistics;
28+
heapSpaceStatistics: HeapSpaceStatistics[];
29+
};
30+
}>;

‎packages/server/tsconfig.json

+2-1
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,8 @@
1515
"noImplicitAny": false,
1616
"declaration": true,
1717
"strict": true,
18-
"resolveJsonModule": true
18+
"resolveJsonModule": true,
19+
"sourceMap": true
1920
},
2021
"include": [
2122
"src"

‎tsconfig.json

+1
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525
/* Type Checking */
2626
"declaration": true,
2727
"strict": true,
28+
"sourceMap": true,
2829
/* Enable all strict type-checking options. */
2930
"typeRoots": ["./node_modules/@types", "./src/types"]
3031
},

0 commit comments

Comments
 (0)
Please sign in to comment.