Pipe to Channel hangs on "Parser: IN_PACKETBEFORE (expecting 16)" #870
Comments
|
Can you also provide both the |
|
Also, what node version are you using? |
Of course. Logs available at https://gist.github.com/ad-m/e360f5b21ac4d88780b861e59e41ab8c .
I used Node I will provide report for working version 12.x to assess which version of Node the problem first appeared. |
|
The report was generated faster than I was expected. Here is data: First failing version is 12.9.0. Changelog mention few changes in area of streams:
Are container images for the "nightly" NodeJS releases available somewhere or do I need to rely on official, compiled files (https://nodejs.org/download/nightly/) and develop them myself? Unless the information provided is sufficient to diagnose the problem? |
|
I don't have experience with containers so I can't answer about that. If you're feeling particularly up to it, you could narrow down which node commit caused the failures by doing a |
It takes a long time to build Node. I will try to use nightly releases first to limit the number of commits in which the bug may have been introduced.
Is the information provided enough to reproduce the problem? I provided client & server code.
Yes. |
|
@mscdex [[e2a2a3f4dd]] - stream: use lazy registration for drain for fast destinations (Robert Nagy) #29095 ssh2 - 0.8.9 We are also using an archive library stream piped to the sftp stream. I created an example case only using ssh2 lib. I had to play around with the highWaterMark on the input stream to cause the failure. (WriteStream.prototype.writev?). Failure on more recent nodes versions is a hang/stall. Works on node 12.8.1 and earlier. Example Code
const fs = require('fs');
const path = require('path');
const { Client } = require('ssh2');
const fileSize = 1024 * 1024;
const zipName = 'test.zip';
const fileName = 'test.txt';
const destPath = '/';
const filePath = path.join(__dirname, fileName);
console.log(`Creating file... ${fileName}`);
fs.writeFileSync(filePath, 'a'.repeat(fileSize));
function runSftpTest() {
const conn = new Client();
conn.on('ready', () => {
console.log('Sftp connected...');
const fileStream = fs.createReadStream(filePath, { highWaterMark: 16 });
conn.sftp(async (err, sftp) => {
if (err) throw err;
const sftpStream = sftp.createWriteStream(path.join(destPath, zipName));
sftpStream.once('finish', () => {
console.log('Sftp finished upload - SUCCESS');
conn.end();
});
fileStream.pipe(sftpStream);
});
}).connect({
host: 'host',
port: 22,
username: 'user',
password: 'pass',
// timing bug? so enabling debug may cause it to work
// debug: console.log,
});
}
const readStream = fs.createReadStream(filePath, { highWaterMark: 16 });
const writeStream = fs.createWriteStream(`${filePath}.new`);
writeStream.once('finish', () => {
console.log(`File stream successful... created ${fileName}.new`);
runSftpTest();
});
readStream.pipe(writeStream); |
|
+1 |
|
@smarusa I have no idea and probably won't have time to investigate unfortunately. I'm currently in the middle of a rewrite which may remedy many of these issues though. |
|
If this is still an issue with |
Hello,
I'm trying to pipe tar archive via ssh to "tar" started by "exec".
When piping data to "exec" channel, the transmission hangs after about 4-5 MB (variable).
The problem occurs both when using ssh2-server and OpenSSH as a server. When performing similiar operation using openssh-client over ssh2-server I don't notice any issues:
cat /tmp/a.tar | ssh $USER@$HOST tar xvf - -C /data/public. Therefore, it seems to me that the problem is on the client side.Whene connection hangs, it only receives information about
Outgoing: Writing REQUEST_FAILURE. Propably for keepalive purpose.I tried to limit the transfer speed using
stream-throotle. This did not bring any significant improvement.I tried ssh2
0.8.7and"github:mscdex/ssh2#b5aad43272ee08509e93e3ae3969c2ccfa1d9d84"I use "tar" & exec because transfer of individual files via sftp was too slow.
I reviewed previous issues - I did not find any similar.
I attach log with timestamp & example source code.
Debug log
Example code
The text was updated successfully, but these errors were encountered: