Synchronous result stream processing

[dendrochronology]

I'm having trouble finding clear examples of how to perform a query and process the result stream synchronously. I've tried the following example from this comment but thePromise hangs, so the script never executes the next line, and node just sits there without exiting (never reaches the connection close line either):

const thingsWeCareAbout = new Map();

await new Promise((resolve, reject) => {
  const s1 = connection.query('SELECT a lot of stuff').stream();
  s1.on('result', row => {
    // logic to determine if we care about this row
    thingsWeCareAbout.set('foreignKeyFromQuery', ['stuff', 'to', 'process', 'later']);
  });
  s1.on('end', resolve);
  s1.on('error', reject);
});

doMoreStuff(thingsWeCareAbout); // this never executes; script is hung

I need to pull millions of rows, but don't want to load the whole thing into memory. I also can't move on to the next step until all the rows have been examined by a function that determines if we need to do something in the next step (it's a chain of ETL functions for a data warehouse).

[dendrochronology]

A colleague suggested removing .stream() from the query, and it works! Did I miss that in the docs somewhere?

[sidorares]

Because you are not composing streams there is no point in using .stream()

I think what happens is no one reads from other side of the stream and it pauses itself ( backpressure handling ) -

node-mysql2/lib/commands/query.js

Line 258 in 80c55ef

if (!stream.push(row)) {

In order to just read rows without buffering them in memory .query() with no callback and 'result' events handler is enough