Connection errors out with long running queries

[sshantt]

Describe the bug
Whenever I run a long running (>1 minute) query, the execution errors out and shows the following message:

Error fetching records. Connection got disposed.

The message is shown 40 seconds into the query running.

To Reproduce
Steps to reproduce the behavior:

1. Connect to a database
2. Create a query that takes longer than 4 minutes to execute and/or contains results in the millions of records and 90+ columns
3. Run the query
4. Wait for the message to pop up

Expected behavior
The query should execute successfully regardless of the result size or the runtime

Desktop (please complete the following information):

• SQLTools Version: v0.20.2
• VSCode Version: 1.38.0-insider
• Dialect Used/Version: Redshift
• OS: Mac

[mtxr]

Did you try to increase connection timeout? By default is ~30

[sshantt]

Yes I did set the connect timeout to a couple of minutes and then tried an hour

[sshantt]

I think this is related

{ "header": { "event": "Allocation failed - JavaScript heap out of memory", "trigger": "FatalError", "filename": "report.20190920.181924.17795.0.001.json", "dumpEventTime": "2019-09-20T18:19:24Z", "dumpEventTimeStamp": "1569003564313", "processId": 17795, "cwd": "/home/username/Code", "commandLine": [ "/home/username/.vscode-server/bin/575062278d181c075ea8dce345ab3d20a127309c/node", "/home/username/.vscode-server-insiders/extensions/mtxr.sqltools-0.20.3/languageserver.js", "--node-ipc", "--clientProcessId=17734" ], "nodejsVersion": "v12.4.0", "glibcVersionRuntime": "2.27", "glibcVersionCompiler": "2.17", "wordSize": 64, "arch": "x64", "platform": "linux", "componentVersions": { "node": "12.4.0", "v8": "7.4.288.27-node.18", "uv": "1.29.1", "zlib": "1.2.11", "brotli": "1.0.7", "ares": "1.15.0", "modules": "72", "nghttp2": "1.38.0", "napi": "4", "llhttp": "1.1.3", "http_parser": "2.8.0", "openssl": "1.1.1b", "cldr": "35.1", "icu": "64.2", "tz": "2019a", "unicode": "12.1" }, "release": { "name": "node", "headersUrl": "https://nodejs.org/download/release/v12.4.0/node-v12.4.0-headers.tar.gz", "sourceUrl": "https://nodejs.org/download/release/v12.4.0/node-v12.4.0.tar.gz" }, "osName": "Linux", "osRelease": "4.15.0-1048-aws", "osVersion": "#50-Ubuntu SMP Wed Sep 4 22:03:52 UTC 2019", "osMachine": "x86_64", "host": "ip-123-45-67-89" }, "javascriptStack": { "message": "No stack.", "stack": [ "Unavailable." ] }, "nativeStack": [ { "pc": "0x0000000000aadaf5", "symbol": "report::TriggerNodeReport(v8::Isolate*, node::Environment*, char const*, char const*, std::string const&, v8::Local<v8::String>) [/home/username/.vscode-server/bin/575062278d181c075ea8dce345ab3d20a127309c/node]" }, { "pc": "0x000000000098fec3", "symbol": "node::OnFatalError(char const*, char const*) [/home/username/.vscode-server/bin/575062278d181c075ea8dce345ab3d20a127309c/node]" }, { "pc": "0x0000000000b1552e", "symbol": "v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/home/username/.vscode-server/bin/575062278d181c075ea8dce345ab3d20a127309c/node]" }, { "pc": "0x0000000000b158a9", "symbol": "v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/home/username/.vscode-server/bin/575062278d181c075ea8dce345ab3d20a127309c/node]" }, { "pc": "0x0000000000f20105", "symbol": " [/home/username/.vscode-server/bin/575062278d181c075ea8dce345ab3d20a127309c/node]" }, { "pc": "0x0000000000f2aa6b", "symbol": "v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/home/username/.vscode-server/bin/575062278d181c075ea8dce345ab3d20a127309c/node]" }, { "pc": "0x0000000000f2b787", "symbol": "v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/home/username/.vscode-server/bin/575062278d181c075ea8dce345ab3d20a127309c/node]" }, { "pc": "0x0000000000f2e225", "symbol": "v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/home/username/.vscode-server/bin/575062278d181c075ea8dce345ab3d20a127309c/node]" }, { "pc": "0x0000000000ef9250", "symbol": "v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/home/username/.vscode-server/bin/575062278d181c075ea8dce345ab3d20a127309c/node]" }, { "pc": "0x00000000011cb0ce", "symbol": "v8::internal::Runtime_AllocateInNewSpace(int, unsigned long*, v8::internal::Isolate*) [/home/username/.vscode-server/bin/575062278d181c075ea8dce345ab3d20a127309c/node]" }, { "pc": "0x0000000001a93942", "symbol": " [/home/username/.vscode-server/bin/575062278d181c075ea8dce345ab3d20a127309c/node]" } ], "javascriptHeap": { "totalMemory": 2149916672, "totalCommittedMemory": 2147688848, "usedMemory": 2144685224, "availableMemory": 50207928, "memoryLimit": 2197815296, "heapSpaces": { "read_only_space": { "memorySize": 524288, "committedMemory": 32024, "capacity": 523976, "used": 31712, "available": 492264 }, "new_space": { "memorySize": 2097152, "committedMemory": 1342616, "capacity": 1047952, "used": 299464, "available": 748488 }, "old_space": { "memorySize": 2135023616, "committedMemory": 2134783504, "capacity": 2133252040, "used": 2133231440, "available": 20600 }, "code_space": { "memorySize": 688128, "committedMemory": 356288, "capacity": 339744, "used": 339744, "available": 0 }, "map_space": { "memorySize": 1576960, "committedMemory": 1167888, "capacity": 838080, "used": 838080, "available": 0 }, "large_object_space": { "memorySize": 9957376, "committedMemory": 9957376, "capacity": 9941328, "used": 9941328, "available": 0 }, "code_large_object_space": { "memorySize": 49152, "committedMemory": 49152, "capacity": 3456, "used": 3456, "available": 0 }, "new_large_object_space": { "memorySize": 0, "committedMemory": 0, "capacity": 1047952, "used": 0, "available": 1047952 } } }, "resourceUsage": { "userCpuSeconds": 54.9049, "kernelCpuSeconds": 4.38025, "cpuConsumptionPercent": 3.33813, "maxRss": 2299691008, "pageFaults": { "IORequired": 0, "IONotRequired": 830667 }, "fsActivity": { "reads": 0, "writes": 0 } }, "uvthreadResourceUsage": { "userCpuSeconds": 31.292, "kernelCpuSeconds": 2.75998, "cpuConsumptionPercent": 1.91734, "fsActivity": { "reads": 0, "writes": 0 } }, "libuv": [ ], "environmentVariables": { "IS_NODE_RUNTIME": "0" }, "userLimits": { "core_file_size_blocks": { "soft": 0, "hard": "unlimited" }, "data_seg_size_kbytes": { "soft": "unlimited", "hard": "unlimited" }, "file_size_blocks": { "soft": "unlimited", "hard": "unlimited" }, "max_locked_memory_bytes": { "soft": 16777216, "hard": 16777216 }, "max_memory_size_kbytes": { "soft": "unlimited", "hard": "unlimited" }, "open_files": { "soft": 1048576, "hard": 1048576 }, "stack_size_bytes": { "soft": 8388608, "hard": "unlimited" }, "cpu_time_seconds": { "soft": "unlimited", "hard": "unlimited" }, "max_user_processes": { "soft": 125378, "hard": 125378 }, "virtual_memory_kbytes": { "soft": "unlimited", "hard": "unlimited" } }, "sharedObjects": [ "linux-vdso.so.1", "/lib/x86_64-linux-gnu/libdl.so.2", "/lib/x86_64-linux-gnu/librt.so.1", "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "/lib/x86_64-linux-gnu/libm.so.6", "/lib/x86_64-linux-gnu/libgcc_s.so.1", "/lib/x86_64-linux-gnu/libpthread.so.0", "/lib/x86_64-linux-gnu/libc.so.6", "/lib64/ld-linux-x86-64.so.2", "/lib/x86_64-linux-gnu/libnss_files.so.2", "/lib/x86_64-linux-gnu/libnss_dns.so.2", "/lib/x86_64-linux-gnu/libresolv.so.2" ] }

[sshantt]

Maybe pulling the first 50 or so records and querying for more whenever a user clicks next on the results page might fix this. Pulling millions of records all at once might be overloading some system resource.

[mtxr]

This is a common issue. It's more like a problem on VSCode language server. See:

microsoft/live-share#245
julia-vscode/julia-vscode#430
reasonml-editor/vscode-reasonml#247

I've found issues about it everywhere. I'll try to contact someone in the VSCode team and see what we can do about that.

[mtxr]

They have some on going effort to avoid this. Since I update language server and client every time they release a new version, I'll close this issue since there's nothing I can do for now.

Maybe pulling the first 50 or so records and querying for more whenever a user clicks next on the results page might fix this. Pulling millions of records all at once might be overloading some system resource.

This is already tracked in #365