Async method call in MySFTPServer __init__ method

[mertpp]

Hello
I need to get the path of the chroot of the user from the database. This is how I use my sftp server to create server.

    await asyncssh.create_server(
        MySSHServer, '', PORT,
        server_host_keys=['/Users/mertpolat/.ssh/id_rsa'],
        process_factory=lambda: None,  # No shell support
        sftp_factory=lambda chan: MySFTPServer(chan, db),  
        allow_scp=True,
        kex_algs=[
            'curve25519-sha256', 'curve25519-sha256@libssh.org', 'ecdh-sha2-nistp256',
            'ecdh-sha2-nistp384', 'ecdh-sha2-nistp521', 'diffie-hellman-group14-sha1'
        ],
        encryption_algs=[
            'aes128-ctr', 'aes192-ctr', 'aes256-ctr'
        ],
        mac_algs=[
            'hmac-sha2-256', 'hmac-sha2-512'
        ],
        compression_algs=['none']
    )

This is how I implemented sftp server. the problem is at the get account by username method. That method should be asynchronous. I couldn't make init method asynchronous.

Copilot suggested me to create a create factory method that returns the instance of mysftpserver. But in that case the code block above doesn't work.

class MySFTPServer(asyncssh.SFTPServer):
    def __init__(self, chan: asyncssh.SSHServerChannel, db: 'Database'):
        self._chan = chan
        self._db = db
        self._username = chan.get_extra_info('username')
        super().__init__(chan)
        username = self._username
        logger.debug(f"Fetching account data for username: {username}")
        home_folder = await self._db.get_account_by_username(username)
        print(f"Account data: {home_folder}")
        if home_folder:
            os.makedirs(home_folder, exist_ok=True)
            logger.info(f"Establishing root_folder for {username} at: {home_folder}")
            self.set_chroot(home_folder)
        else:
            logger.error(f"Account for username {username} not found in the database or data is incomplete.")
            raise ValueError(f"Account for username {username} not found in the database or data is incomplete.")   

How can I change chroot of the sftp user retrieved from the database during the initialization?

I see ssh classes has connection_made method but SFTP subclasses doesn't, so I cannot change the users chroot after initialization.

thank you for your help.

[ronf]

Yeah - this might be tricky at the moment, as both the server_factory and sftp_factory arguments right now only support callables, and don't allow awaitables.

One thing you might try is setting an acceptor on the call to create_server(). That can be an awaitable, and receives an SSHServerConnection as an argument. You can get the username from that connection object via get_extra_info(), similar to what the current example does using chan.get_extra_info(), do your database lookup, and then store the directory information back into the SSHServerConnection with set_extra_info(). Later, in the SFTP initialization, you'd just query for the directory via chan.get_extra_info() instead of querying for the username.

A cleaner fix would be to add the ability in AsyncSSH to use awaitables for more of the factory calls, and I've been doing that a little at a time (most recently to auth and kex exchange calls), but that has its own challenges, and can introduce race conditions if not done very carefully.

[ronf]

As an aside, I also noticed you are setting process_factory here, but I don't believe that should be necessary. AsyncSSH should automatically block shell/exec requests when process_factory and session_factory are not set, unless you allow it via custom callbacks in an SSHServer class. Passing in a lambda which takes no arguments might even break things, as a process factory is supposed to take an SSHServerProcess argument when it is called.

[mertpp]

Hello Thank you very much for your answer I tried with CoPilot and Azure AI and ChatGPT and couldn't succeed. Can you please help me what am I missing here?

import asyncio
import asyncssh
import logging
import os
import sys
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
from db.instance import Database  

logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s')
logger = logging.getLogger(__name__)

USER_HOME = "/Users/mertlat/Downloads/Application4/mertoston/"
AUTHORIZED_KEYS_PATH = os.path.join(USER_HOME, "authorized_keys")
SERVER_HOST_KEYS = ['/Users/mertlat/.ssh/id_rsa']
PORT = 8026

class MySFTPServer(asyncssh.SFTPServer):
    def __init__(self, chan: asyncssh.SSHServerChannel):
        self._chan = chan
        self._home_folder = chan.get_connection().get_extra_info('home_folder')
        super().__init__(chan)

        logger.info(f"Establishing root folder for SFTP at: {self._home_folder}")
        os.makedirs(self._home_folder, exist_ok=True)
        self.set_chroot(self._home_folder)

async def acceptor(conn: asyncssh.SSHServerConnection, db):
    username = conn.get_extra_info('username')
    logger.debug(f"Fetching account data for username: {username}")
    
    # Perform the database lookup to get the home folder
    home_folder = await db.get_account_by_username(username)
    
    if home_folder:
        logger.info(f"Home folder for {username} is: {home_folder}")
        # Store the home folder in the connection object
        conn.set_extra_info('home_folder', home_folder)
        return True
    else:
        logger.error(f"No home folder found for {username}")
        return False

async def start_server(db):
    await asyncssh.create_server(
        MySSHServer, '', PORT,
        server_host_keys=SERVER_HOST_KEYS,
        sftp_factory=MySFTPServer,
        allow_scp=True,
        kex_algs=[
            'curve25519-sha256', 'curve25519-sha256@libssh.org', 'ecdh-sha2-nistp256',
            'ecdh-sha2-nistp384', 'ecdh-sha2-nistp521', 'diffie-hellman-group14-sha1'
        ],
        encryption_algs=[
            'aes128-ctr', 'aes192-ctr', 'aes256-ctr'
        ],
        mac_algs=[
            'hmac-sha2-256', 'hmac-sha2-512'
        ],
        compression_algs=['none'],
        acceptor=lambda conn: acceptor(conn, db)  # Set the acceptor here
    )

class MySSHServer(asyncssh.SSHServer):
    def __init__(self):
        pass

    def connection_made(self, conn):
        peername = conn.get_extra_info('peername')
        logger.info(f'Connection made from {peername} to {conn.get_extra_info("sockname")}')

    def connection_lost(self, exc):
        if exc:
            logger.error(f'Connection lost with error: {exc}')
        else:
            logger.info('Connection closed')

    def begin_auth(self, username):
        logger.info(f'Beginning authentication for {username}')
        return True

    def password_auth_supported(self):
        return False  # Deshabilitar autenticación por contraseña

    def public_key_auth_supported(self):
        return True  # Habilitar autenticación por clave pública

    async def validate_public_key(self, username, key):
        logger.info(f'Validating public key for {username}')
        try:
            if not os.path.exists(AUTHORIZED_KEYS_PATH):
                logger.error(f'Authorized keys file not found: {AUTHORIZED_KEYS_PATH}')
                return False

            with open(AUTHORIZED_KEYS_PATH, 'r') as f:
                authorized_keys = f.read().splitlines()
                logger.info(f'Authorized keys loaded: {authorized_keys}')
                for auth_key in authorized_keys:
                    logger.debug(f'Checking key: {auth_key}')
                    if key == asyncssh.import_public_key(auth_key):
                        logger.info(f'Public key authentication successful for {username}')
                        return True
        except IOError as e:
            logger.error(f'Failed to read authorized keys for {username}: {e}')
        except asyncssh.KeyImportError as e:
            logger.error(f'Failed to import public key: {e}')
        logger.warning(f'Public key authentication failed for {username}')
        return False


async def start_sftp_server():
    logger.info('Starting SFTP server...')

    if not os.path.exists(AUTHORIZED_KEYS_PATH):
        logger.error(f'Authorized keys file not found at startup: {AUTHORIZED_KEYS_PATH}')
    else:
        logger.info(f'Authorized keys file found at startup: {AUTHORIZED_KEYS_PATH}')

    db = Database()
    await db.init_pool()

    await start_server(db)

    logger.info('SFTP Server started on port 8026')
    await asyncio.Event().wait()  # Run the server indefinitely

if __name__ == "__main__":
    try:
        asyncio.run(start_sftp_server())
    except (OSError, asyncssh.Error) as exc:
        logger.error(f'Error starting server: {exc}')
        sys.exit('Error starting server: ' + str(exc))

[ronf]

I see two things that look like problems. First, the call to set_extra_info needs to look something like:

        conn.set_extra_info(home_folder=home_folder)

Second, your init() code in MySFTPServer should look more like the original chroot example. For instance:

    def __init__(self, chan: asyncssh.SSHServerChannel):
        self._chan = chan
        self._home_folder = chan.get_connection().get_extra_info('home_folder')
        os.makedirs(self._home_folder, exist_ok=True)
        super().__init__(chan, chroot=self._home_folder)

        logger.info(f"Establishing root folder for SFTP at: {self._home_folder}")

This is adding the chroot to the super()__init__() rather than trying to set it separately. There is no set_chroot() function on SFTPServer.

[mertpp]

Thank you for your fast answer. Following your recommendations I could progress more but it still ground zero:

os.PathLike object, not NoneType because basically the info is empty.

  File "/Users/mertpolat/Documents/Projects/MFTKoyunco/koyunco/core/venv/lib/python3.12/site-packages/asyncssh/stream.py", line 717, in _init_sftp_server
    return self._sftp_factory(cast('SSHChannel[bytes]', self._chan))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mertpt/Documents/Projects/MFTKoyunco/koyunco/core/bin/server.py", line 23, in __init__
    os.makedirs(self._home_folder, exist_ok=True)
  File "<frozen os>", line 210, in makedirs
  File "<frozen posixpath>", line 103, in split
TypeError: expected str, bytes or os.PathLike object, not NoneType
2024-10-06 20:16:59,931 - INFO - [conn=0, chan=0] Closing channel due to connection close
2024-10-06 20:16:59,931 - ERROR - Connection lost with error: expected str, bytes or os.PathLike object, not NoneType

It's like there is no way to pass this information to MySFTPServer.

[ronf]

I didn't run into that here, but that suggests to me that maybe the database lookup failed and returned None, or perhaps that the set_extra_info() still isn't quite right and it's not putting the home folder into the extra info properly. You might want to try adding some logging around those, and maybe even try doing a get_extra_info() call directly from the acceptor right after you set the home folder, to see if you can fetch it from there.

[kiskele]

I have a similar issue, it might be because acceptor is started with create_task() and result is not awaited, so init of sftp_factory is started before acceptor saved home_directory into extra_info

[ronf]

Good point. A quick workaround would be to make the acceptor argument a callable instead of an awaitable to make it run synchronously, but that would block the entire event loop while the database lookup happened in this example. The other option would be to explicitly synchronize the two operations to make sure the acceptor finishes first, but that's tricky to do from the SFTP constructor, as that's not async and right now the chroot() needs to happen when the SFTP server object is constructed. Relaxing that would risk some requests coming in before the chroot has been set up.

Looking at the code more closely, it might be good enough here to put the database lookup into an async begin_auth() method on your SSHServer subclass. That will get the username as an argument, and while auth isn't complete at that point, it would allow you to set the user's home directory into the extra info from a callback which is awaited, and later if auth succeeds and you end up creating an SFTP server instance, it will have what it needs.

Now that the auth code supports more async callbacks, it would be possible to await on the acceptor instead of just firing it off in the background, but that might break existing client code if someone had an acceptor function which didn't return quickly, counting on the fact that it is currently run in the background. A better option would probably be to add code to allow the auth_completed() callback to be async, and find a way to await on that, leaving the acceptor untouched. That would require some re-arranging of the code as the current auth_completed() callback is done from a synchronous function, but that shouldn't be difficult to rework.

[kiskele]

Yes, my solution is also to run the async DB query in SSHServer.begin_auth. The problem is that I don't have access to SSHServerConnection from SSHServer. It was very-very painful to create a hacked version of SSHServerConnection, but the best place to move data from SSHServer to extra_info was in SSHServerConnection.send_userauth_success. So maybe an option to provide factory for SSHServerConnection would be a big help in the future

[ronf]

AsyncSSH is designed to have all custom code in SSHClient/SSHServer and SSHSession objects. SSHConnection and SSHChannel objects aren't meant to be customized. This is modeled after asyncio's Transport and Protocol classes.

You get access to the SSHServerConnection from the SSHServer through the connection_made() callback. When the TCP connection is accepted, an SSHServerConnection will be created and passed to the SSHServer via connection_made(). You can store that in a member of your custom SSHServer class for use later, such as when the begin_auth() method is called.

[ronf]

Commit e827d13 in the "develop" branch now makes the SSHServer auth_complete() callback capable of being either a callable or a coroutine. It runs after auth completes successfully, with the username available in the SSHServerConnection's get_extra_info() (as "username"), and it runs to completion before any of the session/process/sftp factories are instantiated, avoiding the race condition you saw with the acceptor.

The acceptor still runs as an independent task and is not awaited on, preserving backward compatibility. The auth_complete() callback will run to completion before the acceptor is started (as well as before other factories are started), so all of those can depend on any data filled in during this callback.

[ronf]

This feature is now included in AsyncSSH 2.21.0.