Unable to upload in ASCII mode

[molekamp]

In March we upgraded from System.Net.FtpClient to version 16.3.0 of FluentFTP and since then we are seeing a problem that seems very similar to Issue #122. We use the method public virtual Stream OpenWrite(string path, FtpDataType type) with data type ASCII, both before and after the upgrade. We write bytes to the stream and then close it. What we are seeing is that transfers of smaller files still work as normal, but for larger files (taking over 15 seconds to transfer) our client is suddenly sending a TCP RESET to the server before all data has been sent. Then it continues to send the remainder of the data, but that no longer reaches the other end, leaving a file that is incomplete. The remote server is of type VMS.
When the problem occurs, we see this error in our logs: System.TimeoutException: Timed out trying to read data from the socket stream!

We tried to fix this by changing to the high level interface method:
public bool Upload(byte[] fileData, string remotePath, FtpExists existsMode = FtpExists.Overwrite, bool createRemoteDir = false)
Although this succeeds in getting the whole file across, the FtpDataType used is now Binary. This leads to the files being unreadable. They must be sent in ASCII mode in order to be converted to EBCDIC on the target mainframe. The high level API offers no way to choose ASCII data type.
The Upload methods that take a byte[] or Stream both call UploadFileInternal, which in turn always calls the low level methods OpenWrite or OpenAppend that only take a path argument. Both of these then call the overload taking a path and FtpDataType and set the data type to Binary.

All of our attempts to use the low level methods to achieve ASCII transfer have failed, leading to the same time out error described at the beginning.

We also tried upgrading to the latest version from NuGet, 17.5.0, but this had no effect.

We expect the high level API to offer a way to set the data type to ASCII.

[robinrodricks]

First lets try debugging OpenWrite as its used internally for high level APIs too. No changes have been made to OpenWrite from System.Net.FtpClient (before, after). No changes have been made to the data stream class either (before, after). TCP reset, you say? Maybe its timing out, can you try explicitly setting the timeouts specifically DataConnectionReadTimeout which might help. Did you try OpenWrite with System.Net.FtpClient? Did it work or not?

Second the high level API does not support ASCII transfers but it should be trivial to implement. I'll attempt a version today and get back to you with a debug version.

[robinrodricks]

I think there is a bug with the timeouts, ie you need to explicitly set their values, the defaults are not taking effect.... Can you try setting all the timeouts like shown in the troubleshooting entry for Azure?

[molekamp]

We specifically see the failure after 15 seconds, which is also the default value for all timeouts, so it is indeed very likely to be related.

[molekamp]

Until this is fixed, our workaround is to do some of the low level stuff ourselves like so:

        private bool UploadFile(string path, string body, FtpClient ftpClient)
        {
            try
            {
                using (Stream writeStream = ftpClient.OpenWrite(path, FtpDataType.ASCII))
                {
                    var transferChunkSize = 65536;
                    var stream = new MemoryStream(Encoding.UTF8.GetBytes(body));
                    int readBytes;
                    var buffer = new byte[transferChunkSize];

                    // Start sending data
                    while ((readBytes = stream.Read(buffer, 0, buffer.Length)) > 0)
                    {
                        // Write chunk to the FTP stream
                        writeStream.Write(buffer, 0, readBytes);
                        writeStream.Flush();
                    }

                    while (writeStream.Position < writeStream.Length)
                    {
                        // Waiting for data to be sent.
                        Thread.Sleep(10);
                    }
                    // All data sent
                }
                FtpReply reply = ftpClient.GetReply();
                if (!reply.Success)
                {
                    // Log ERROR {reply.ErrorMessage}, {reply.Code}
                }
                else
                {
                    //Log INFO {reply.InfoMessages}, {reply.Code}
                }
                return reply.Success;
            }
            catch (IOException ex)
            {
                // Log Exception
                return false;
            }
        }

[robinrodricks]

Can you try increasing all timeouts to a very large value and check if that solves it?

[robinrodricks]

Can you try this beta version, which has new properties (UploadDataType, DownloadDataType) that control usage of the ASCII/Binary mode with the high level API.

• FluentFTP_17.5.3_BETA1.zip

[molekamp]

Can't test right now unfortunately, but looking at the code the fix to allow ASCII/Binary mode looks good.

Thinking some more about our old code, the problem could also be that we never called GetReply() after OpenWrite(). Is writing to the FtpDataStream asynchronous? In that case it may be that it got garbage collected before it even finished.

[robinrodricks]

It's not async, you just need to call GetReply else there will be stale data on the socket... The reply must be read..

When this fix works for you I'll push it to nuget.

[robinrodricks]

Can you test it and get back to me so I can do a final build?

[robinrodricks]

I'm closing this as I've added ASCII support to the high level APIs. Feel free to message here if you encounter any more issues.

[gattamanenim]

Hi,
I am trying to upload a file using FluentFTP upload method. but I am getting this error.
"NLST command invalid with TYPE binary"

Any suggestions Please?