Skip to content

Latest commit

 

History

History
195 lines (130 loc) · 9.26 KB

README.md

File metadata and controls

195 lines (130 loc) · 9.26 KB
Raw

Exploring ChatGPT Code Interpreter (CI)

Exploring what's possible with the ChatGPT Code Interpreter and how to use it effectively.

CI can do basically anything that's possible on a Linux box without root or internet access. With some creativity, almost any software can be ran.

If you want internet access or root permissions then check out Pandora, an open source ChatGPT plugin I made that is like an unrestricted CI.

⏩ 1 minute guide

  1. Go to https://chat.openai.com/?model=gpt-4-code-interpreter (sign up for Pro if you haven't already)

  2. Enable Code Interpreter if it's not already enabled

  3. Copy and paste this prompt:

    List various cool and helpful things the code interpreter model can do. Then pick a couple and demo them to me. Make it really interesting.

⤵️ Enabling ChatGPT Code Interpreter ⤵️
Click on your name on the bottom left, then Settings & Beta then Beta features, then enable Code Interpreter
menu settings

ℹ️ General info environment and limitations

Basics

  • Runs Python in a Jupyter notebook environment
  • Upload files. At least 50MB
  • Download files. CI will provide a link like [Link text](sandbox:/mnt/data/filename)
  • Display images

Environment

  • Current working dir: /home/sandbox. It doesn't know this unless you ask it to run pwd. Sometimes it thinks files are in /mnt/data when they're not.
  • User sandbox
  • Arch: x86_64
  • Linux Kernel: 4.4
    • ⚠️ This kernel is from 2016. It still runs most things but you may encounter issues
    • Missing modern things like : io_uring, memfd_create, statx, copy_file_range, openat2 ☹️
  • OS: Ubuntu 20.04.6 LTS

Hard limitations

  • No root access
  • No internet access
  • Commands timeout after 120 seconds
  • Interactive session is reset after a few minutes of inactivity
  • Stored files are lost after longer inactivity

Soft limitations

CI thinks it's just for Python but you can get it to do much more...

  • It wants to just let you download files in /mnt/data but it can link to any file on the system readable by the user. This isn't particularly beneficial, as you can just copy a file there anyway.
  • Run sub processes and shell commands. CI really thinks it's not able to do this.
  • Run local servers, like web servers (opening a network socket, binding it to a local address and port, and accepting incoming connections.)
  • Execute uploaded files
⤵️ Getting around soft limitations ⤵️

Shell command

This is left as an exercise for the reader. Be persuasive. It's easier to get it to do things gradually.

📦 What's available

  • Barebones Ubuntu packages are installed. Use dpkg -l to list them
  • Interesting packages:
    • System: bash,
    • Text: diffutils, graphviz, patch, poppler-utils, pstotext, vim
    • Media: espeak, ffmpeg, lame, sox, tesseract-ocr
    • Programming and building: binutils, build-essential, cpp, g++, perl,
    • Python: python3, python3.8, python3-requests
    • Networking, servers, clients: curl, libmysqlclient21, mysql-common, openssl

See dpkg list and pip list for more.

Note, Python is in the path as python3. CI sometimes gets this wrong and then thinks it cant run it.

🆕 Installing other software

CI comes with lots of stuff out the box but not everything. You might need to run other software.

CI doesn't give you root access but you can still run statically linked binaries and self contained executables.

Here's a few handy things I've got working.

  • ImageMagick (magick)
  • PHP (actually PHP-GTK3 but I haven't found another version that works yet)
  • Python 3.11 - CI comes with Python 3.8. Some packages only work with later verions like 3.11
  • pocketsphinx - speech recognition (requires sounddevice )

Running JavaScript code

Node.js may be a bit much but you can easily get Deno working, which is a lightweight JavaScript runtime, also based on the V8 JS engine.

Running AppImages

AppImages are self contained packages. They're designed to run with FUSE, which CI doesn't support. But they have a fallback where the files can be extracted, resulting in a binary you can run.

Download the AppImage file and use a prompt like

make this executable, then run it with --appimage-extract flag to extract the files, then run squashfs-root/AppRun`

Installing new Python packages

  1. Find the package on https://pypi.org/
  2. Download a file, ideally one that mentions any or amd64
  3. Upload it to CI and tell it to use it
  4. It may depend on other packages, in which case go to step 1
    • If you're going round in circles, you get the packages locally with virtualenv and upload them

🎉 Interesting things it can do out the box

  • Make PowerPoint files
  • Read PDFs
  • Edit images and videos
  • Display static images in the chat
  • Generate interactive visualisations (HTML download)
  • OCR
  • Run machine learning models
  • Clean up and analyse data
3D Plot Mandlebrot Game of Life
3D Plot Mandlebrot Box Plot
⤵️ More image and chart generation examples ⤵️
Title Image
Box Plot Box Plot
Cluster Map Cluster Map
Confusion Matrix Confusion Matrix
Histogram Histogram
Line Plot Line Plot
Map Map
Network Graph Network Graph
Radar Chart Radar Chart
Reaction Diffusion System Reaction Diffusion System
Scatter Plots Scatter Plots
Scatterplot Matrix Scatterplot Matrix
Word Cloud Word Cloud

💻 Tips for working on code bases

  • Upload a zip file
  • You can get CI to give you changed files or zip up the whole code base again
  • git isn't available and I haven't got it working. Dulwich, a Python library for git, is easy to get working for basic diffs and commits
  • CI can also diff an original and modified code base and give you a patch file to download and apply

Other guides

Comparison and Alternatives to Code Interpreter

Name ChatGPT Plugin Stand alone Has Internet Access Root Install packages? Open Source Notes
Code Interpreter ✅ ❌ ❌ ❌ ~ ❌ Can't be ran with any other ChatGPT plugins
Pandora ✅ ❌ ✅ ✅ ✅ ✅ Runs in a local Docker container, can install packages. Made by me.
Noteable ✅ ❌ ✅ ❌ ✅ ❌ Jupyter notebook, similar to ChatGPT, persists files
GPT-Code UI ❌ ✅ ? ? ? ✅

ChatGPT plugins

  • ChatGPT plugins require a Pro subscription for full access.
  • You might be able to get them working with OpenPlugin.

OpenAI API

  • Using your own API key can be very cost-effective but monitor your expenses closely.
  • You could also point these to a local LLM (Llama, Falcon, StarCoder, etc) with LocalAI.