Create the new aio-local-ai-cuda12 image based on nvidia-cuda12 locai-ai docker image

[luzfcb]

WIP pull-request

docker compose build cuda12

Notes:

• This requires at least 46GB of disk space and probably 46GB of free space when update. In other words, it is safe to say that you will need at least 100GB of disk space to use this image.

• The free Github Actions runner does not have enough disk space to build this docker image, even if we use techniques to remove unused tools such as Android SDK, .net and any other non-docker tools.

• Alternatively, the Circle-ci free account has 150GB of free disk space, which was enough to build the docker image and publish it to docker hub. https://hub.docker.com/repository/docker/luzfcb/aio-local-ai-cuda12/tags

• It not support ARM Architecture.

• It is possible to configure circle-ci to be started via HTTP request directly via Github Actions instead of a git action by using the https://github.com/marketplace/actions/trigger-circleci-pipeline.

This allows having Github Actions as a launcher, making maintenance and control a little easier.

[szaimen]

Hi, thanks a lot for working on this and creating this PR!

Unfortunately I will not be able to support or maintain this as I do not own an Nvidia gpu. Thus I will not merge this.

So better would be if you could either take over this project or to fork it and do your modifications on top of it, publish the docker image on docker hub and create and additional community container by using this as template but modifying it accordingly: https://github.com/nextcloud/all-in-one/blob/main/community-containers/local-ai/local-ai.json

Also see https://github.com/nextcloud/all-in-one/tree/main/community-containers#how-to-add-containers

Let me know if you need any help on this :)

[luzfcb]

I do not own an Nvidia gpu.

I honestly think Nextcloud GmbH should provide the Nvidia hardware for you, since you are the most active developer on AIO and I believe that AI will inevitably be of interest to them.

That said, I guess perhaps this could be resolved by the community in a few weeks or less if a crowdfunding campaign was launched specifically to fund the acquisition of an Nvidia GPU for you.

Maybe I think you might not be interested in having yet another project to maintain and that's perfectly acceptable... It's sad, but being an open source software maintainer generates a lot of unpaid and unrecognized work.

to fork it and do your modifications on top of it, publish the docker image on docker hub and create and additional community container by using this as template but modifying it accordingly: https://github.com/nextcloud/all-in-one/blob/main/community-containers/local-ai/local-ai.json

I prefer to follow the fork approach.

The question I currently have is how to test locally the complete integration flow with Nextcloud AIO, without necessarily having previously created a new folder containing a .json in https://github.com/nextcloud/all-in-one/tree/main/community-containers and/or it already being included at least in a new beta release of AIO.

Is there already a guide on this? Since Nextcloud AIO controls docker, starts and deletes containers, the process of integrating other containers is still a bit hazy for me.

I compiled and published to docker hub the first attempt, but I don't know if it works.

docker pull luzfcb/aio-local-ai-cuda12:v2

https://hub.docker.com/repository/docker/luzfcb/aio-local-ai-cuda12/tags

[luzfcb]

nextcloud/all-in-one#5795

[szaimen]

I do not own an Nvidia gpu.

I honestly think Nextcloud GmbH should provide the Nvidia hardware for you, since you are the most active developer on AIO and I believe that AI will inevitably be of interest to them.

That said, I guess perhaps this could be resolved by the community in a few weeks or less if a crowdfunding campaign was launched specifically to fund the acquisition of an Nvidia GPU for you.

Maybe I think you might not be interested in having yet another project to maintain and that's perfectly acceptable... It's sad, but being an open source software maintainer generates a lot of unpaid and unrecognized work.

Yes, this is actually the main reason. Thanks for your understanding!

to fork it and do your modifications on top of it, publish the docker image on docker hub and create and additional community container by using this as template but modifying it accordingly: https://github.com/nextcloud/all-in-one/blob/main/community-containers/local-ai/local-ai.json

I prefer to follow the fork approach.

Thanks a lot!

The question I currently have is how to test locally the complete integration flow with Nextcloud AIO, without necessarily having previously created a new folder containing a .json in https://github.com/nextcloud/all-in-one/tree/main/community-containers and/or it already being included at least in a new beta release of AIO.

Is there already a guide on this? Since Nextcloud AIO controls docker, starts and deletes containers, the process of integrating other containers is still a bit hazy for me.

Not exactly, but you can mount the new folder that you created in /var/www/docker-aio/community-containers via e.g. -v ./your-folder:/var/www/docker-aio/community-containers/your-folder/ and then actiate the container via --env AIO_COMMUNITY_CONTAINERS="your-folder" during the docker run command.

I hope this helps :)

[szaimen]

nextcloud/all-in-one#5795

Thanks ☺️