Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Yade SMB transfer fails after 60min #10

Open
rfx77 opened this issue Nov 6, 2023 · 5 comments
Open

Yade SMB transfer fails after 60min #10

rfx77 opened this issue Nov 6, 2023 · 5 comments
Assignees
@sosap
Labels
question

Comments

@rfx77
Copy link

rfx77 commented Nov 6, 2023

Hi!

I have a file Move-Job which moves a file from a SFTP Source to a SMB Target. When the move runs longer than 60min ist fails with an smbj error. The file is 20 to 30GB in size.

When the jobs runs under 60min it always succeeds

i am on version 2.6.2

2023-10-31 00:35:45.755+01:00 [MAIN] [OrderProcessingStarted] id=#2023-10-30#P10380987302-Scada.Sch.Backup.Ign01, Job=moveBackup, pos=5, Agent(url=http://agent:4443, name=docker-primary, time=2023-10-31 00:35:45.635+01:00)
2023-10-31 00:35:45.635+01:00 [MAIN]    [Start] Job=moveBackup, Agent (url=http://agent:4443, id=docker-primary, name=docker-primary)
2023-10-31 00:35:46.055+01:00 [STDOUT]   
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+                       START  : YADE.SH                          +
+                       ----------------                          +
+ DATE     : Mon Oct 30 23:35:45 UTC 2023                         +
+ HOSTNAME : 9394d9181bac                                         +
+ USER     :                                                      +
+ CALL     : /opt/js7/agent/yade/bin/yade.sh -settings /opt/js7/agent/data/work/values/0/364.xml -profile move-backup -return-values "/opt/js7/agent/data/work/tmp/returnValues-5061675402264897796.tmp" +
+                                                                 +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
 
2023-10-31 00:35:47.956+01:00 [STDOUT]  2023-10-30T23:35:47,553 INFO  main c.s.D.SOSDataExchangeEngineMain - SOSDataExchange - Main routine started ...
2023-10-30T23:35:47,567 INFO  main c.s.D.SOSDataExchangeEngine - 
************************************************************************
*                                                                      *
*                     YADE - Managed File Transfer                     *
*                     -----www.sos-berlin.com-----                     *
*                                                                      *
************************************************************************
  Version                 = 1.13.19 (2023-07-11 15:12, revision d1f90a8bc2d9df099d452aadc2f92128857873ee) Copyright 2003-2023 SOS GmbH Berlin
  Date                    = 2023-10-30 23:35:47
  SettingsFile            = /opt/js7/agent/data/work/values/0/364.xml
  Profile                 = move-backup
  Operation               = move
  Transactional           = false
  +------------Source------------
  | Protocol              = sftp
  | Host                  = 192.168.26.31
  | IP                    = 192.168.26.31
  | User                  = root
  | AuthMethod            = password
  | Password              = ***
  | FilePath              = /opt/backup/prod/scada-backup-full.tgz
  | ErrorWhenNoFilesFound = true
  | Recursive             = false
  | Remove                = true
  +------------Target------------
  | Protocol              = smb
  | Host                  = 192.168.20.84
  | IP                    = 192.168.20.84
  | User                  = Administrator
  | AuthMethod            = publickey
  | Password              = ***
  | Directory             = /backup/
  | OverwriteFiles        = true
2023-10-30T23:35:47,909 INFO  main c.h.s.c.PacketEncryptor - Initialized PacketEncryptor with Cipher << AES_128_GCM >>
2023-10-30T23:35:47,910 INFO  main c.h.s.c.Connection - Successfully connected to: 192.168.20.84
2023-10-31 00:35:48.389+01:00 [STDOUT]  2023-10-30T23:35:47,988 INFO  main c.h.s.c.SMBSessionBuilder - Successfully authenticated Administrator on 192.168.20.84, session is 105614319553925
2023-10-30T23:35:47,988 INFO  main c.s.v.s.c.ASOSSMB - [smb][xxx.local][Administrator@192.168.20.84:445]
2023-10-30T23:35:47,991 INFO  main c.h.s.s.Session - Connecting to \\192.168.20.84\backup on session 105614319553925
2023-10-30T23:35:48,040 INFO  main c.s.v.s.s.SOSSFTPSSHJ - [sftp]SOSVfs-D-0101: Try to connect to host '192.168.26.31' at Port '22'.
2023-10-30T23:35:48,216 INFO  main n.s.s.t.r.JCERandom - Creating new SecureRandom.
2023-10-30T23:35:48,289 INFO  main n.s.s.t.TransportImpl - Client identity string: SSH-2.0-SSHJ_0.35.0
2023-10-30T23:35:48,298 INFO  main n.s.s.t.TransportImpl - Server identity string: SSH-2.0-OpenSSH_8.9p1 Ubuntu-3ubuntu0.3
2023-10-31 00:35:49.247+01:00 [STDOUT]  2023-10-30T23:35:48,846 INFO  main c.s.D.SOSDataExchangeEngine - [source]1 files found.
2023-10-31 00:43:16.486+01:00 [STDERR]  2023-10-30T23:43:16,438 ERROR main c.h.p.c.c.Promise - << 110523 >> woke to: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: java.net.SocketException: Connection reset
2023-10-30T23:43:16,442 ERROR main c.s.v.c.SOSFileListEntry - SOSVfs_E_229: error. unable to transfer data, reason: com.sos.JSHelper.Exceptions.JobSchedulerException: SOSVfs_E_173: write failed for file /backup/scada-backup-full.tgz (com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: java.net.SocketException: Connection reset)
2023-10-30T23:43:16,442 ERROR main c.s.D.SOSDataExchangeEngine - SOSDataExchangeEngine.TRANSFER_ABORTED
2023-10-30T23:43:16,444 ERROR Packet Reader for 192.168.20.84 c.h.s.s.Session - Caught exception while closing TreeConnect with id: 1
com.hierynomus.protocol.transport.TransportException: java.net.SocketException: Broken pipe
	at com.hierynomus.smbj.transport.tcp.direct.DirectTcpTransport.write(DirectTcpTransport.java:78) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.connection.Connection.send(Connection.java:234) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.session.Session.send(Session.java:300) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.TreeConnect.close(TreeConnect.java:69) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.Share.close(Share.java:116) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.session.Session.logoff(Session.java:236) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.session.Session.close(Session.java:279) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.connection.Connection.close(Connection.java:178) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.connection.Connection.close(Connection.java:155) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.connection.Connection.handleError(Connection.java:294) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.transport.PacketReader.run(PacketReader.java:54) ~[smbj-0.11.5.jar:?]
	at java.lang.Thread.run(Thread.java:1623) [?:?]
Caused by: java.net.SocketException: Broken pipe
	at sun.nio.ch.NioSocketImpl.implWrite(NioSocketImpl.java:418) ~[?:?]
	at sun.nio.ch.NioSocketImpl.write(NioSocketImpl.java:438) ~[?:?]
	at sun.nio.ch.NioSocketImpl$2.write(NioSocketImpl.java:817) ~[?:?]
	at java.net.Socket$SocketOutputStream.write(Socket.java:1120) ~[?:?]
	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:125) ~[?:?]
	at java.io.BufferedOutputStream.implFlush(BufferedOutputStream.java:252) ~[?:?]
	at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:240) ~[?:?]
	at com.hierynomus.smbj.transport.tcp.direct.DirectTcpTransport.write(DirectTcpTransport.java:75) ~[smbj-0.11.5.jar:?]
	... 11 more
2023-10-31 00:43:16.515+01:00 [STDERR]  2023-10-30T23:43:16,484 ERROR main c.s.D.SOSDataExchangeEngineMain - execute: Error occurred ...: SOSVfs_E_173: write failed for file /backup/scada-backup-full.tgz (com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: java.net.SocketException: Connection reset), exit-code 99 raised
com.sos.JSHelper.Exceptions.JobSchedulerException: SOSVfs_E_173: write failed for file /backup/scada-backup-full.tgz (com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: java.net.SocketException: Connection reset)
	at com.sos.vfs.smb.common.SOSSMBFile.write(SOSSMBFile.java:45) ~[com.sos-berlin.jobscheduler.virtual-file-system-1.13.19.jar:1.13.19]
	at com.sos.vfs.common.SOSFileListEntry.doTransfer(SOSFileListEntry.java:230) ~[com.sos-berlin.jobscheduler.virtual-file-system-1.13.19.jar:1.13.19]
	at com.sos.vfs.common.SOSFileListEntry.run(SOSFileListEntry.java:1208) ~[com.sos-berlin.jobscheduler.virtual-file-system-1.13.19.jar:1.13.19]
	at com.sos.DataExchange.SOSDataExchangeEngine.sendFiles(SOSDataExchangeEngine.java:1076) ~[com.sos-berlin.jade.jade-engine-1.13.19.jar:1.13.19]
	at com.sos.DataExchange.SOSDataExchangeEngine.transfer(SOSDataExchangeEngine.java:1358) ~[com.sos-berlin.jade.jade-engine-1.13.19.jar:1.13.19]
	at com.sos.DataExchange.SOSDataExchangeEngine.execute(SOSDataExchangeEngine.java:430) ~[com.sos-berlin.jade.jade-engine-1.13.19.jar:1.13.19]
	at com.sos.DataExchange.SOSDataExchangeEngineMain.execute(SOSDataExchangeEngineMain.java:76) [com.sos-berlin.jade.jade-engine-1.13.19.jar:1.13.19]
	at com.sos.DataExchange.SOSDataExchangeEngineMain.main(SOSDataExchangeEngineMain.java:58) [com.sos-berlin.jade.jade-engine-1.13.19.jar:1.13.19]
Caused by: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: java.net.SocketException: Connection reset
	at com.hierynomus.smbj.share.Share.receive(Share.java:394) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.Share.receive(Share.java:376) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.Share.write(Share.java:204) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.SMB2Writer.write(SMB2Writer.java:103) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.FileOutputStream.sendWriteRequest(FileOutputStream.java:90) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.FileOutputStream.flush(FileOutputStream.java:85) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.FileOutputStream.write(FileOutputStream.java:68) ~[smbj-0.11.5.jar:?]
	at com.sos.vfs.smb.common.SOSSMBFile.write(SOSSMBFile.java:43) ~[com.sos-berlin.jobscheduler.virtual-file-system-1.13.19.jar:1.13.19]
	... 7 more
Caused by: com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: java.net.SocketException: Connection reset
	at com.hierynomus.protocol.transport.TransportException$1.wrap(TransportException.java:30) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.protocol.transport.TransportException$1.wrap(TransportException.java:24) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.protocol.commons.concurrent.Futures.get(Futures.java:47) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.Share.receive(Share.java:389) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.Share.receive(Share.java:376) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.Share.write(Share.java:204) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.SMB2Writer.write(SMB2Writer.java:103) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.FileOutputStream.sendWriteRequest(FileOutputStream.java:90) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.FileOutputStream.flush(FileOutputStream.java:85) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.FileOutputStream.write(FileOutputStream.java:68) ~[smbj-0.11.5.jar:?]
	at com.sos.vfs.smb.common.SOSSMBFile.write(SOSSMBFile.java:43) ~[com.sos-berlin.jobscheduler.virtual-file-system-1.13.19.jar:1.13.19]
	... 7 more
Caused by: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: java.net.SocketException: Connection reset
	at com.hierynomus.protocol.commons.concurrent.PromiseBackedFuture.get(PromiseBackedFuture.java:59) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.protocol.commons.concurrent.CancellableFuture.get(CancellableFuture.java:84) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.protocol.commons.concurrent.Futures.get(Futures.java:42) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.Share.receive(Share.java:389) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.Share.receive(Share.java:376) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.Share.write(Share.java:204) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.SMB2Writer.write(SMB2Writer.java:103) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.FileOutputStream.sendWriteRequest(FileOutputStream.java:90) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.FileOutputStream.flush(FileOutputStream.java:85) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.share.FileOutputStream.write(FileOutputStream.java:68) ~[smbj-0.11.5.jar:?]
	at com.sos.vfs.smb.common.SOSSMBFile.write(SOSSMBFile.java:43) ~[com.sos-berlin.jobscheduler.virtual-file-system-1.13.19.jar:1.13.19]
	... 7 more
Caused by: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: java.net.SocketException: Connection reset
	at com.hierynomus.smbj.common.SMBRuntimeException$1.wrap(SMBRuntimeException.java:28) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.common.SMBRuntimeException$1.wrap(SMBRuntimeException.java:22) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.protocol.commons.concurrent.Promise.deliverError(Promise.java:95) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.connection.OutstandingRequests.handleError(OutstandingRequests.java:88) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.connection.Connection.handleError(Connection.java:292) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.transport.PacketReader.run(PacketReader.java:54) ~[smbj-0.11.5.jar:?]
	at java.lang.Thread.run(Thread.java:1623) ~[?:?]
Caused by: com.hierynomus.protocol.transport.TransportException: java.net.SocketException: Connection reset
	at com.hierynomus.smbj.transport.tcp.direct.DirectTcpPacketReader.doRead(DirectTcpPacketReader.java:53) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.transport.PacketReader.readPacket(PacketReader.java:70) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.transport.PacketReader.run(PacketReader.java:48) ~[smbj-0.11.5.jar:?]
	at java.lang.Thread.run(Thread.java:1623) ~[?:?]
Caused by: java.net.SocketException: Connection reset
	at sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:318) ~[?:?]
	at sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:345) ~[?:?]
	at sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:794) ~[?:?]
	at java.net.Socket$SocketInputStream.read(Socket.java:1025) ~[?:?]
	at com.hierynomus.smbj.transport.tcp.direct.DirectTcpPacketReader.readFully(DirectTcpPacketReader.java:70) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.transport.tcp.direct.DirectTcpPacketReader.readTcpHeader(DirectTcpPacketReader.java:59) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.transport.tcp.direct.DirectTcpPacketReader.doRead(DirectTcpPacketReader.java:48) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.transport.PacketReader.readPacket(PacketReader.java:70) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.transport.PacketReader.run(PacketReader.java:48) ~[smbj-0.11.5.jar:?]
	at java.lang.Thread.run(Thread.java:1623) ~[?:?]
2023-10-31 00:43:16.517+01:00 [STDOUT]  2023-10-30T23:43:16,392 INFO  Packet Reader for 192.168.20.84 c.h.s.t.t.d.DirectTcpPacketReader - PacketReader error, got exception.
com.hierynomus.protocol.transport.TransportException: java.net.SocketException: Connection reset
	at com.hierynomus.smbj.transport.tcp.direct.DirectTcpPacketReader.doRead(DirectTcpPacketReader.java:53) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.transport.PacketReader.readPacket(PacketReader.java:70) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.transport.PacketReader.run(PacketReader.java:48) ~[smbj-0.11.5.jar:?]
	at java.lang.Thread.run(Thread.java:1623) [?:?]
Caused by: java.net.SocketException: Connection reset
	at sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:318) ~[?:?]
	at sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:345) ~[?:?]
	at sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:794) ~[?:?]
	at java.net.Socket$SocketInputStream.read(Socket.java:1025) ~[?:?]
	at com.hierynomus.smbj.transport.tcp.direct.DirectTcpPacketReader.readFully(DirectTcpPacketReader.java:70) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.transport.tcp.direct.DirectTcpPacketReader.readTcpHeader(DirectTcpPacketReader.java:59) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.transport.tcp.direct.DirectTcpPacketReader.doRead(DirectTcpPacketReader.java:48) ~[smbj-0.11.5.jar:?]
	... 3 more
2023-10-30T23:43:16,438 INFO  Packet Reader for 192.168.20.84 c.h.s.s.Session - Logging off session 105614319553925 from host 192.168.20.84
2023-10-30T23:43:16,442 INFO  main c.s.v.c.SOSFileList - Set transfer status
2023-10-30T23:43:16,443 INFO  main c.h.s.s.Session - Logging off session 105614319553925 from host 192.168.20.84
2023-10-30T23:43:16,445 INFO  main c.h.s.s.Session - Logging off session 105614319553925 from host 192.168.20.84
2023-10-30T23:43:16,446 INFO  main c.h.s.SMBClient - Going to close all remaining connections
2023-10-30T23:43:16,447 INFO  main c.h.s.c.Connection - Closed connection to 192.168.20.84
2023-10-30T23:43:16,447 WARN  Packet Reader for 192.168.20.84 c.h.s.c.Connection - Exception while closing session 105614319553925
com.hierynomus.protocol.transport.TransportException: Cannot write Encrypted[SMB2_LOGOFF with message id << 110542 >>] as transport is disconnected
	at com.hierynomus.smbj.transport.tcp.direct.DirectTcpTransport.write(DirectTcpTransport.java:68) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.connection.Connection.send(Connection.java:234) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.session.Session.send(Session.java:300) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.session.Session.logoff(Session.java:256) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.session.Session.close(Session.java:279) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.connection.Connection.close(Connection.java:178) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.connection.Connection.close(Connection.java:155) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.connection.Connection.handleError(Connection.java:294) ~[smbj-0.11.5.jar:?]
	at com.hierynomus.smbj.transport.PacketReader.run(PacketReader.java:54) ~[smbj-0.11.5.jar:?]
	at java.lang.Thread.run(Thread.java:1623) [?:?]
2023-10-30T23:43:16,447 INFO  Packet Reader for 192.168.20.84 c.h.s.c.Connection - Closed connection to 192.168.20.84
2023-10-30T23:43:16,447 INFO  main c.s.v.s.c.ASOSSMB - [smb][xxx.local][Administrator@192.168.20.84:445][disconnect]com.hierynomus.protocol.transport.TransportException: java.net.SocketException: Broken pipe
2023-10-30T23:43:16,449 INFO  main n.s.s.t.TransportImpl - Disconnected - BY_APPLICATION
2023-10-30T23:43:16,450 INFO  main c.s.v.s.s.SOSSFTPSSHJ - disconnect OK
2023-10-30T23:43:16,477 INFO  main c.s.D.SOSDataExchangeEngine - SOSJADE_I_0101: 0 files transferred (2023-10-30 23:35:47-2023-10-30 23:43:16)7m28.893558284s
2023-10-30T23:43:16,484 INFO  main c.s.D.SOSDataExchangeEngine - 
************************************************************************* 
 execution status     = failure. Errors reported. 
 successful transfers = 0 
 skipped transfers    = 0 
 failed transfers     = 1 
 last error           = SOSVfs_E_173: write failed for file /backup/scada-backup-full.tgz (com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: java.net.SocketException: Connection reset)
 
*************************************************************************
 
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+                         END  : YADE.SH                          +
+                         --------------                          +
+ DATE     : Mon Oct 30 23:43:16 UTC 2023                         +
+ HOSTNAME : 9394d9181bac                                         +
+ USER     :                                                      +
+ CALL     : /opt/js7/agent/yade/bin/yade.sh -settings /opt/js7/agent/data/work/values/0/364.xml -profile move-backup -return-values "/opt/js7/agent/data/work/tmp/returnValues-5061675402264897796.tmp" +
+ EXIT     : 99                                                   +
+                                                                 +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
 
2023-10-31 00:43:16.518+01:00 [MAIN]    [End] [Error] returnCode=99, errorState=failed, reason=failed, msg=SOSVfs_E_173: write failed for file /backup/scada-backup-full.tgz (com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: java.net.SocketException: Connection reset)
2023-10-31 00:43:16.612+01:00 [ERROR] [OrderProcessed] id=#2023-10-30#P10380987302-Scada.Sch.Backup.Ign01, Job=moveBackup, pos=5, Agent(time=2023-10-31 00:43:16.518+01:00), Error (status=failed, msg=SOSVfs_E_173: write failed for file /backup/scada-backup-full.tgz (com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: java.net.SocketException: Connection reset)
), returnCode=99
2023-10-31 00:43:16.796+01:00 [ERROR] [OrderFailed] id=#2023-10-30#P10380987302-Scada.Sch.Backup.Ign01, pos=5, Error (status=failed, msg=[Job=moveBackup, pos=5]SOSVfs_E_173: write failed for file /backup/scada-backup-full.tgz (com.hierynomus.protocol.transport.TransportException: java.util.concurrent.ExecutionException: com.hierynomus.smbj.common.SMBRuntimeException: com.hierynomus.protocol.transport.TransportException: java.net.SocketException: Connection reset)
), returnCode=99
@sosap
Copy link
Contributor

sosap commented Nov 6, 2023

If a connection is reset exactly after 60 min. in a reproducible manner then this does not suggest that this limit comes from the YADE file transfer tool. There is no such built-in limit.

Can you please check network connections, possibly a firewall holding a rule for connections timeouts?

@sosap sosap added the question label Nov 6, 2023
@sosap sosap self-assigned this Nov 6, 2023
@rfx77
Copy link
Author

rfx77 commented Nov 6, 2023

There is no firewall between the Agent and the SMB-Server

@sosap
Copy link
Contributor

sosap commented Nov 6, 2023

If you are certain about this then please check Connection reset by peer: socket write error #365, it looks close to what you report.

If the above applies then this suggests

  1. to check if an idle timeout is configured with your SMB Server,
  2. another question: why 20 GB take >60 min. for file transfer? Root cause 1) requires an idle connection, this could indicate that no data have been transferred from the SFTP source for a longer period, This could shift the problem focus from the target to the source.
  3. that there is no immediate solution from the smbj project that didn't pick up this topic for the last 5 years.

Should 1) and 2) apply and should the problem live with the source of transfer then you can use the "server_alive_interval=true" parameter, for XML Configuration the ServerAliveInterval element with the value true, see https://kb.sos-berlin.com/display/PKB/YADE+Parameter+Reference+-+SFTPFragment, that will cause YADE to send keep-alive packets to the source of transfer.

@rfx77
Copy link
Author

rfx77 commented Nov 15, 2023

After some investiagation we found that the 60min are misleading. There seems to bee some sort of Timezone issue in the logs.

From the logs you can see that the transfer fails after some 7 to 8 minutes

2023-10-30T23:43:16,477 INFO main c.s.D.SOSDataExchangeEngine - SOSJADE_I_0101: 0 files transferred (2023-10-30 23:35:47-2023-10-30 23:43:16)7m28.893558284s

@JozefDropco
Copy link

JozefDropco commented Feb 10, 2025 •
edited
Loading

@rfx77 its not weird timezone issue ;) you have 2 types of logs written out.

  1. Regular logs without timezone
  2. STDOUT / STDERR with timezone 2023-10-31 00:43:16.517 +01:00 [STDOUT] 2023-10-30T23:43:16,392 INFO Packet Reader

We have similar application in our work which is copying data from one location to another. We had to implement RecovarableOutputStream on top of SmbClient as SMBJ is doing bare minimum when it comes to SMB protocol. Disaster recovery is not part of it. OutputStream is opened and transferred - in ideal world works great. In real scenarios where unpredictable restarts, issues on the network path might "break" the pipe - those are not covered at all. Broken pipe however in our scenario meant we were using DFS but in older versions of SMBJ it didnt work well during resolution of the "real" server location

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sosap sosap

Labels
question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@JozefDropco @sosap @rfx77