Handling images

[adityakavalur]

This is a great project, that captures a lot of the workflows that folks might need in day-to-day command line actions. I have been playing around with it a little and found it super helpful.

However, one workflow that doesn't work for me through sgpt is image handling. If GPT were to create a plot using matplotlib, I'd like for it to be able to see it as well. GPT 4o has vision capabilities (relevant info here), it looks like you need to pass type as image_url, and then either provide the image url if its accessible or provide the image using base64 encoding. My initial thoughts are adding a handler for this in addition to the repl and chat handler might enable this functionality.

I am wondering if this is something that is being looked at or do folks think its beyond the scope of the project to incorporate this capability?