Virtual Memory (VIRT) shooting up when using the latest version of npm

[CookieBOY]

🐛 Bug Report

Upgrading the package major version from 7 to 8 shoots the virtual memory by 10~11 GB

To Reproduce

Steps to reproduce the behaviour:

const elasticsearch = require('@elastic/elasticsearch')
const express = require('express')

var app = express()

app.listen(9000)

Running the above node code and just checking the virtual ram using top or netstat

Expected behaviour

As using the elasticsearch version 7 npm package the virtual memory is is around 1gb expecting the same behaviour with version 8

Paste the results here:

Environment

• node version: 14
• @elastic/elasticsearch version: >=8.0.0
• os: Linux

[JoshMock]

Possibly related to elastic/elastic-transport-js#63. Will take a look soon.

[JoshMock]

Just realized this is concerning virtual memory. As noted here, virtual memory allocation is not bound to available physical memory, so it isn't a helpful metric for monitoring memory usage.

To see what the actual memory heap usage was between 7.x and 8.x using your example code, I modified it to print out heap usage every 5 seconds for 20 minutes:

const elasticsearch = require('@elastic/elasticsearch')
const express = require('express')

let timer = 0
setInterval(() => {
  console.log(process.memoryUsage().heapUsed)
  timer++
  if (timer > 240) { // run for 20 minutes
    process.exit()
  }
}, 5000)

var app = express()

app.listen(9000)

I ran this with the 7.17.0 client and the 8.8.0 client, using Node.js v14, and charted memory usage for each, in MB:

Here's the same metrics, but running on Node.js v18:

So, the 8.x client does use a bit more memory than the 7.x client at startup, but a 0.5MB difference in Node v14 (or 2MB in Node v18) is a trivial difference unless you're working on very constrained hardware.

So there's not really anything to be done here, unless you have examples of heap memory usage exploding in a similar way. Hope that clears some things up!