App exits silently when downloading 52 image links concurrently

[xanhz]

Version

v18.18.2

Platform

Linux dev 5.15.0-89-generic #99~20.04.1-Ubuntu SMP Thu Nov 2 15:16:47 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

Here is my code and my machine specs:

• axios: v1.5.1
• RAM: 16GB
• Disk: 512GB SSD
• CPU: Intel® Core™ i5-10400 CPU @ 2.90GHz × 12

const { default: axios } = require('axios')
const { randomUUID } = require('crypto')
const fs = require('fs')
const path = require('path')

async function main() {
    try {
        const links = [] // 50 -> 60 images with size in [10, 80] MB
        const promises = links.map(link => download(link))
        await Promise.allSettled(promises)
    } catch (error) {
        console.error(error)
    }
}

async function download(link) {
    console.log('Downloading Link=%s', link)
    const response = await axios.get(link, { responseType: 'stream' })
    const filepath = path.join(__dirname, 'tmp', randomUUID())
    const writeStream = fs.createWriteStream(filepath)
    console.log('Writing Link=%s into Path=%s', link, filepath)
    return new Promise((resolve, reject) => {
        response.data
            .pipe(writeStream)
            .on('error', error => reject(error))
            .on('finish', () => {
                console.log('Downloaded successfully Link=%s', link)
                resolve()
            })
    })
}

main()

How often does it reproduce? Is there a required condition?

Always

What is the expected behavior? Why is that the expected behavior?

I expected that it will download successfully all links or throw error. But when about 40 files was written into disk, my app was stopped with status 0.

What do you see instead?

This is my log file. I was replace links and file path by other text because of security. You can count number of 'Downloading' is 54, 'Writing' is 54 and 'Downloaded' is 40
foo.log

Additional information

No response

[marco-ippolito]

I highly suspect it has to do with your code, Node.js empties its event loop and has no additional work to schedule.
Try add to add

process.on("beforeExit", function () {
 console.log('beforeExit')
});

And see if it is invoked 😄
As there is no reason to believe this is a bug inside node itself, I'm going to go ahead and close out the issue, feel free to reopen

[xanhz]

@marco-ippolito yes, i realize that the reason is pipe does not throw error when error occurs on readstream or writestream. So I modify my code as below:

const { default: axios } = require('axios')
const { randomUUID } = require('crypto')
const fs = require('fs')
const path = require('path')

async function main() {
    try {
        const links = [] // 50 -> 60 images with size in [10, 80] MB
        const promises = links.map(link => download(link))
        await Promise.allSettled(promises)
    } catch (error) {
        console.error(error)
    }
}

async function download(link) {
    console.log('Downloading Link=%s', link)
    const response = await axios.get(link, { responseType: 'stream' })
    return new Promise((resolve, reject) => {
        const filepath = path.join(__dirname, 'tmp', randomUUID())
        const writeStream = fs.createWriteStream(filepath)
        writeStream.on('error', error => {
            console.log('Writestream error', error)
            reject(error)
        })
        console.log('Writing Link=%s into Path=%s', link, filepath)
        response.data
            .on('error', error => {
                console.error('Readstream error', error)
                reject(error)
            })
            .pipe(writeStream)
            .on('error', error => reject(error))
            .on('finish', () => {
                console.log('Downloaded successfully Link=%s', link)
                resolve()
            })
    })
}

main()

It throws error

Readstream error Error: aborted
    at connResetException (node:internal/errors:717:14)
    at TLSSocket.socketCloseListener (node:_http_client:462:19)
    at TLSSocket.emit (node:events:525:35)
    at node:net:322:12
    at TCP.done (node:_tls_wrap:588:7) {
  code: 'ECONNRESET'
}