Race conditions when requesting shortened URLs

[johnlinp]

Describe the bug

• Node.js version: 20.10.0
• OS & version: Ubuntu 22.04.5 LTS

When making requests to various types of URLs, we see some failures for shortened URLs. The failures are not consistent, so I assume there is some race conditions going on.

The error logs look like this:

Nov 20 23:41:08 info: got-scraping called; parameters: (url:'https://fcld.ly/5hdnik1', ...) 
Nov 20 23:41:08 file:///app/node_modules/got/dist/source/core/index.js:446 
Nov 20 23:41:08                 if (this._request._writableState?.errored) { 
Nov 20 23:41:08                                   ^   
Nov 20 23:41:08 TypeError: Cannot read properties of undefined (reading '_writableState') 
Nov 20 23:41:08     at ClientRequest.<anonymous> (file:///app/node_modules/got/dist/source/core/index.js:446:35) 
Nov 20 23:41:08     at Object.onceWrapper (node:events:628:28) 
Nov 20 23:41:08     at ClientRequest.emit (node:events:526:35) 
Nov 20 23:41:08     at request.emit (file:///app/node_modules/got-scraping/dist/index.js:434:14) 
Nov 20 23:41:08     at onFinish (node:_http_outgoing:1005:10) 
Nov 20 23:41:08     at afterWrite (node:internal/streams/writable:693:5) 
Nov 20 23:41:08     at onwrite (node:internal/streams/writable:671:7) 
Nov 20 23:41:08     at WriteWrap.onWriteComplete [as oncomplete] (node:internal/stream_base_commons:106:10) 
Nov 20 23:41:08     at WriteWrap.callbackTrampoline (node:internal/async_hooks:130:17) 

Actual behavior

The call fails inconsistently (sometimes successful, sometimes fails)

Expected behavior

The call should be successful

Code to reproduce

The code is calling got-scraping (https://github.com/apify/got-scraping), but I think the problem lies in got.

const response = await gotScraping({
    url: 'https://fcld.ly/5hdnik1',
    throwHttpErrors: true,
    timeout: {
        request: 20000
    }
});

Checklist

• I have read the documentation.
• I have tried my code with the latest version of Node.js and Got.