Introducing GitHub Models

[Akash1134]

Alt Text: The graphic is a promotional banner for the introduction of "GitHub Models." It features a dark purple background with various AI model names listed with the GitHub logo at the center.

We are enabling the rise of the AI engineer with GitHub Models–bringing the power of industry leading large and small language models to our more than 100 million users directly on GitHub.

Every developer can be an AI engineer with the right tools and training. From playground to coding with the model in Codespaces to production deployment via Azure, GitHub Models shows you how simple it can be. We invite you to be among the first to experience GitHub Models by signing up for the waitlist. By joining the waitlist, you'll have the opportunity to test and provide feedback on this innovative tool, helping us make them even better.

GitHub.Models.mp4

From Llama 3.1, to GPT-4o and GPT-4o mini, to Phi 3 or Mistral Large 2, you can access each model via a built-in playground that lets you test different prompts and model parameters, for free, right in GitHub. Discover more details in the full blog post by GitHub CEO, Thomas Dohmke.

We can’t wait to see what you will build from here. Sign up now!🚀

Resources to keep you up-to-date:

• GitHub Models blog post
• GitHub Models Demo video
• GitHub Models Docs
• Sign up for the waitlist
• Share your feedback!

[Simon1059770342]

If the application is successful, can I call the API of these large language models for free? Or only provide local deployment tutorials and original models?

[rajbos]

@Simon1059770342 : check the rate limit section on the docs here. So definitely not intended for production use.

[RDjarbeng]

Does anyone get a message like this when trying to use the meta-llama models from their sample code?

{ code: 'unauthorized', message: 'Unauthorized', details: null }

Sorry if this isn't the right place to post this.

[RDjarbeng]

Just saw this.

Sorry for the messages, it seems I'm on the waitlist, that could explain the error

[Akash1134]

Thank you for asking that question. Once you are approved of the waitlist, you should be access the listed models 🙂

[001123]

Awesome! I hope I can be accepted to the waitlist. 🙏

[001123]

Yay, I accepted, and I am very excited to try it.

[machar94]

@Akash1134 , unless stated otherwise, can we assume that the models are all FP32? No quantization? I specifically curious about Meta-Llama-3-70B-Instruct.

[emimrulkayes]

I am on the waiting list. I don't know how long I will wait here? But I am very excited to test it.

[silverkiwi]

Also on waitlist ... avg wait time I heard was a few hours if you're successful, otherwise it's like waiting for Sora.

[Kishlay-notabot]

I've been accepted! happened a few days ago... So I don't think it would take too long.

[emimrulkayes]

I've been accepted! happened a few days ago... So I don't think it would take too long.

Congratulations 🎊

[7086cmd]

I submitted the application for it approximately eight weeks ago, but it has not yet been accepted. Could you please provide me with an update on the status of my application?

[emimrulkayes]

Yeah! finally I've been accepted! happened a few days ago.

[khairu-aqsara]

I have tested several models with the Continue Dev VsCode Plugin, and everything seems to be working well.

[prime399]

how did you configured, I don't see any option for using github model's API in continue plugin?

[maoling1411]

Everything is great, but the TPD and the 8192 input limit are really too low for me as a heavy AI user.

[RDjarbeng]

Still waiting after a month, to get off the waitlist

[yashksaini-coder]

I always wanted to build an application using Open AI but it required API keys, for which I don't have the means to purchase. but finally with Github, I can access various models for completely free.

Alright can anyone suggests some basic ideas to build for a complete beginner to practice once we get the access for the Github Models ?

[JuneReyP]

Hi, you can actually use the free tier for open AI

[yashksaini-coder]

Wow Thanks for this how do i get that @JuneReyP

[Inglan2]

I think its like a free trial

[mendhak]

Could you/anyone share some use cases for this, or something they're envisioning?

I understand the playground aspect, I've been experimenting with a few of the models. I like that the github token can be used here which makes it easy to call from a Github Action, for example.

[BaseMax]

Good as a start: https://www.youtube.com/watch?v=WiBB8Lsgl7I

[imotai]

I encounter the following error When I use the model API with oauth2 token.

{"error":{"code":"unauthorized","message":"Integration auth is not supported for this endpoint","details":null}}

Does this error mean I can only use the model API with PAT( Persional Access Token)?

[vikashkodati]

Cool. just got access. looking into it this weekend

[singhhimanshu97]

GitHub models are great in every aspect. The latest models with beta versions are excellent in terms of performance regarding speed as well as quality of answers.

[filipesilva]

Hey I think you have a typo (conpletions) on the video code sample:

[lqduy]

I would like to be accepted on the waitlist. Thank you!

[FaheemOnHub]

Guys as we have gained access to these models, how can we fine-tune or prompt engineer them or add retrieval-augmented generation ?

[rajbos]

@FaheemOnHub , have you looked at the samples in the Codespace? See a RAG example here: https://github.com/github/codespaces-models/blob/main/cookbooks/python/llamaindex/rag_getting_started.ipynb

Finetuning is not available as the offering to GitHub Models does not store any index or user data to fine tune with. That's also more compute intensive and therefore production like in my opinion. Perhaps that it is added in the future, who knows?

[azman0101]

I receive a mail "You've been granted access to the limited public preview of OpenAI o1 models on GitHub!"
But when I trying to use it, I'm asked to join the waitlist.

[dnafication]

What are the conditions for using these marketplace models? I'm aware that they are intended for trial purposes and not for production use. Could you provide information on the privacy policies? Is it permissible to test these models with private data? Also, do you monitor or have access to the usage of these models?

[dnafication]

Just found them:

Subject to Product Terms & Privacy Statement.