-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
No more extra pay for chatgpt please #138
Comments
Thanks for trying aider! I have not yet seen evidence that any other model can code as well as GPT-4. Aider has experimental support for hooking up to local models. This FAQ entry has more information: https://aider.chat/docs/faq.html#can-i-use-aider-with-other-llms-local-llms-etc |
We have a model like wizardcoder-guanaco which is a great coding model. Its able to understand the context and create code very well and has 8k ctx. I have been using it and had very great success where lot of the code works on first try. There is even a code completion extention for vscode for this model |
Here is a Proof of Concept for you to test using a model hosted locally or on Google colab
Then install it with Next create a file called llama_2_13b_chat_ggml.ipynb and place the contents below in the file
Next connect to google colab Make sure to follow the progress by scrolling down. You need to use that for aider when you run it. make sure you are in a git enabled folder and run aider with the following command (Make sure to update the --openai-api-base=https://fs-authors-studying-carlo.trycloudflare.com/v1 with the line provided by the model in Colab) Once you get to the calculator.c prompt enter the following as a test. The result after a few minutes should be an updated calculator.c that you can compile and test. NOTE: as Aider was not written for this model it may not always work as expected but this test case works just fine. MAKE SURE TO DISCONNECT AND DELETE THE RUNTIME ON GOOGLE COLAB ONCE DONE |
Thanks, this seems to work. A bit slower than OpenAI but its free, so. What do you mean by this:
|
Hmm, it didn't actually write the code, but did print a diff, so it "kinda worked".
|
Ah, I was pointing at the system aider. I had to put this in my git aider-llama2 clone parent directory and I called it free_aider and ran it as a executable called ./free_aider e.g.:
|
@apcameron check out the test project. Did you have this problem with file names? https://github.com/funkytaco/react-test-aider-LLaMa2 |
This is just a reminder to delete the runtme and disconnect from google colab as they only give you a limited time to use each day. If you leave it may potentially use up all your time for the day |
Yes it does not always get the filenames correct but the calculator example always works correctly for me. The aider code may need updating to be compatible with LLama2 or we may need to find a better model |
@paul-gauthier Have you had a chance to try this? |
I've done a bunch of testing of aider against the following models - it wasn't too hard to get them running (using textgen-webui) and 'the hax' from @tmm1 https://huggingface.co/TheBloke/Llama-2-7b-Chat-GPTQ Short version is they were all too underpowered to be useful I got an ok hello world out of wizard but then instructions to modify it would generate new files instead of edits to the old file (it wouldn't understand the instructions to only edit the previous file). I quickly got beyond its ability with instructions like saying hello to the first arg passed (which would error if no arg passed in) and pasting the error message in to chat would just get it to acknowledge there was a problem, congratulate me for finding it and make no effort to actually fix it. My impression at this stage is you want to test against at least a 30b plus model (downloading a few now) to get one which can understand the file instructions. From what I've seen so far, I don't think tweaking with the prompts will get much traction on the smaller models. |
@joshuavial Perhaps you can try this one [huggingface.co/TheBloke/WizardCoder-Guanaco-15B-V1.1]https://huggingface.co/TheBloke/WizardCoder-Guanaco-15B-V1.1-GPTQ) |
I'll add it to the list - I'm currently planning on TheBloke/starcoderplus-GPTQ and TheBloke/FreeWilly2-GPTQ I'm also planning on setting up a runpod VPS to test out the 70b llama-2 at some stage. |
Its a pity that Petals does not have an Openai api yet I opened request petals-infra/chat.petals.dev#20 |
FYI, we just added an #llm-integrations channel on the discord, as a place to discuss using aider with alternative or local LLMs. |
Hmm, well an update. Basic hack: I just typed this as my first line: I removed that flag to print the SYSTEM text. Maybe I'll take another look when I'm not tired. |
Closing this for now. See #172 for more info. |
not all of us use discord |
Not everyone used IRC in the 90's, either. Those people learned another way. |
LOL 90S? we used ICQ. And before that nothing. We programmed on commodore 64s or ICON's with Watcom and Apple II or IIgs to everything in between to where we are now I can adapt Im just sick of switching back forth and having my personal info on a tonne of sites with new potential for creepy peeps while I need to ask questions. And when Discord first appeared I saw it on dark web as major form communication other than protonmail or scary chatrooms I would never go into as I am not stupid and obv not born yesterday lol. but it seems ok so Im trying it. |
i wonder has anyone tried ctransformers AutoTokenizer, AutoModelForCausalLM model "bigcode/starcoderplus" |
description sounds ok |
maybe not, but im going to try a few report back - am I just old or does anyone else feel guilty consulting GPT 4 on fine details of replacing itself |
Hi, just wondering with the new open sources coming out like llama2 or any other at that level i hugginface why are we still using API from openai???... If i can operate at the same level as GT4 using an open source why are we still using openain? ... Counting ofcourse im having them download locally as well.
The text was updated successfully, but these errors were encountered: