Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"ConnectionResetError: Connection lost" occurs when disk space is 100% used. #182

Open
tvvister opened this issue Jan 29, 2025 · 2 comments

Comments

@tvvister
Copy link

tvvister commented Jan 29, 2025

In case when space is not enough on the server side the stream.write operation fails with ConnectionResetError.
msg is like that 'Connection lost', and that's it, nothing is useful at all:

from aioftp import Client


async def main():
    client = Client()
    await client.connect(
        '0.0.0.0',
        port=21,
    )
    await client.login(
        user='',
        password='*',
    )
    try:
        stream = await client.upload_stream('test.txt', offset=0)
        await stream.write(b'data' * 1000)
    except ConnectionResetError:
        prepare_next_try()

Is there way to recognize that the problem is related to the shortage of volume rather than network issue?
If there is, then it would be helpful to make a stop of uploading instead of keep trying to reconnect endlessly.

Here is an example which shows something similar i suppose to have:

    try:
        stream = await client.upload_stream('test.txt', offset=0)
        await stream.write(b'data' * 1000)
    except ConnectionResetError:
        prepare_next_try()
    except NoSpaceError:
        stop_uploading()
@pohmelie
Copy link
Collaborator

Yes, it is possible, but should be implemented in 3 places:

  • PathIO level
  • Server level
  • Client level
    There are some return codes, which represents such cases:
         452 Requested action not taken.
             Insufficient storage space in system.
         552 Requested file action aborted.
             Exceeded storage allocation (for current directory or
             dataset).

https://datatracker.ietf.org/doc/html/rfc959

If I will find a time for this I will implement this. Anyway 3rd person PR would be good.

@tvvister
Copy link
Author

tvvister commented Feb 1, 2025

I've tried to check server response for popular ftp servers, which run from docker.

  • garethflowers/ftp-server
  • delfer/alpine-ftp-server

Both have broken pipeline without any excuse. I had only this:

BrokenPipeError(32, 'Broken pipe')

Nothing about error with codes 552 or 452. (upd.)

Also i found that the final exception will differ from time to time. I use 100Mb partition for ftp volume, and vary initial files' set and chunk size before start.

import aioftp
import asyncio

async def main():
    client = aioftp.Client()
    await client.connect("0.0.0.0")
    await client.login("user", "1234")

    async with client.upload_stream('/bytes1.txt') as out:
        while True:
            await out.write(b'0' * 65536) 
            # await out.write(b'0' * 25_000_000) # ~25Mb

There are two types of final exceptions:

  • File "/home/user/.local/share/python3.10/lib/python3.10/asyncio/streams.py", line 167, in _drain_helper raise ConnectionResetError('Connection lost')
  • File "/home/user/.local/share/python3.10/lib/python3.10/asyncio/selector_events.py", line 949, in _write_ready n = self._sock.send(self._buffer) BrokenPipeError: [Errno 32] Broken pipe

Could someone specify which impl of ftp server is full enough to handle disk full (452, 552 error codes) carefully?

updated: i found the way to adjust an ftp server to send me a response of 552 error code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants