Skip to content

Latest commit

 

History

History
85 lines (73 loc) · 4.2 KB

README.md

File metadata and controls

85 lines (73 loc) · 4.2 KB
Raw

RunpodSetup

  • MixPix AI uses runpod serverless for executing ComfyUI Workflow.

Basic Setup Overview

  • First create a network drive in Runpod using a GPU based VM.
  • Then perform the initial setup like intalling ComfyUI, Virtual Env, Hugging face model etc.
  • Once that is done you can use serverless functions to mount your network drive to perform on demand workflows.

Detailed Steps (WIP)

  • [Step-1] First create a Network drive in Runpod. I am using 100GB network drive.

    • Go to https://www.runpod.io/console/user/storage
    • Tap on New Network Volume
    • Choose your Data Center, Name and Size (Note you will not be able to edit the size later. Might want to use somewhere between 75GB to 100GB).

  • [Step-2] Next go to https://www.runpod.io/console/pods

    • Tap on + GPU Pod
    • Tap on Select Network Volume and choose your Network Volume you created in the above step.
    • Tap on one of the available Machines based on your cost affordability.
    • Tap on Type to search for a template and choose a template that works for you. I am using RunPod Pytorch 2.1(runpod/pytorch:2.1.0-py3.10-cuda11.8.0-devel-ubuntu22.04 )
    • Next Customize Deployment, Expose TCP Ports ==> 22, 8188, Click Set Overrides. This will expose port 8188 so that you can use ComfyUI without tunneling.
    • [Optional] Choose Spot(Interruptible) if you want to spend less cost and don't mind interruptions.
    • Hit Continue

  • [Step-3] [Initial first time only] Next Launch the Jupiter Notebook and do the initial setup

    • Open terminal
    • python -m venv venv
    • source venv/bin/activate
    • git clone https://github.com/mixpixai/RunpodSetup.git
    • mv RunpodSetup/* .
    • mv essentials/* .
    • pip install runpod
    • ipython comfyui_colab_v2.ipynb

  • [Step-4] [Making your workflow] If you need to make a new workflow or add some models you can follow these steps.

    • Create your Pod as mentioned in Step-2
    • Open Jupiter terminal
    • source venv/bin/activate
    • ./init.sh
    • Go to Pods Dashboard
    • Tap on Connect > TCP Port mappings.
    • Check your public IPAddress and the port-id for 8188 and open it in new tab. Something like http://69.30.85.113:22139/
    • Now create your workflow if needed. Test if the basic prompt is working and generating the image.
    • To download the workflow as api. Tap on Settings in ComfyUI. Enable Developer mode.
    • Download Save (API format). This will download it in API format.

  • [Step-5] Create your Serverless Template

    • Go to https://www.runpod.io/console/user/templates
    • Tap New Template
      • Give some name
      • Choose serverless in template Type
      • Add your container Image. I am using runpod/pytorch:2.1.0-py3.10-cuda11.8.0-devel-ubuntu22.04
      • Container Start command bash -c "cd /runpod-volume; bash /runpod-volume/start_serverless.sh"
      • Container Disk 15 GB
      • Hit Save Template

  • [Step-6] Create a new serverless endpoint

    • Go to https://www.runpod.io/console/serverless
    • Tap + New Endpoint
      • Give some endpoint name eg: workflow_api
      • Choose your preferred GPU workers.
      • Confirm Active workes are 0 and Max workers are 3 or less.
      • Keep idle timeout to be 300 or less.
      • Enable Flashbot
      • Continer Configuration - Select the template from Step-5
      • IMP - Advanced - Select your network volume.
      • Scale Type - You can choose Queue Delay or Request Count.

  • [Step -7] Get API key

    • Go to https://www.runpod.io/console/user/settings and expand API_KEY
    • Now tap on + API Key
    • Create a new Read API key and copy it.

  • [Step-8] Now make the curl call

    • curl -H "Content-Type: application/json" -H "Authorization: Bearer <YOUR API KEY>" <YOUR SERVERLESS ENDPOINT> -d @request_body.json
    • eg: curl -H "Content-Type: application/json" -H "Authorization: Bearer INTL56110******" https://api.runpod.ai/v2/q47mbggytrqq9i/runsync -d @request_body.json

Now your serverless endpoint is ready. NOTE: sometimes the first call might fail.