Introducing GitHub Models

[Akash1134]

Alt Text: The graphic is a promotional banner for the introduction of "GitHub Models." It features a dark purple background with various AI model names listed with the GitHub logo at the center.

We are enabling the rise of the AI engineer with GitHub Models–bringing the power of industry leading large and small language models to our more than 100 million users directly on GitHub.

Every developer can be an AI engineer with the right tools and training. From playground to coding with the model in Codespaces to production deployment via Azure, GitHub Models shows you how simple it can be. We invite you to be among the first to experience GitHub Models by signing up for the waitlist. By joining the waitlist, you'll have the opportunity to test and provide feedback on this innovative tool, helping us make them even better.

GitHub.Models.mp4

From Llama 3.1, to GPT-4o and GPT-4o mini, to Phi 3 or Mistral Large 2, you can access each model via a built-in playground that lets you test different prompts and model parameters, for free, right in GitHub. Discover more details in the full blog post by GitHub CEO, Thomas Dohmke.

We can’t wait to see what you will build from here. Sign up now!🚀

Resources to keep you up-to-date:

• GitHub Models blog post
• GitHub Models Demo video
• GitHub Models Docs
• Sign up for the waitlist
• Share your feedback!

[Simon1059770342]

If the application is successful, can I call the API of these large language models for free? Or only provide local deployment tutorials and original models?

[rajbos]

@Simon1059770342 : check the rate limit section on the docs here. So definitely not intended for production use.

[RDjarbeng]

Does anyone get a message like this when trying to use the meta-llama models from their sample code?

{ code: 'unauthorized', message: 'Unauthorized', details: null }

Sorry if this isn't the right place to post this.

[RDjarbeng]

Just saw this.

Sorry for the messages, it seems I'm on the waitlist, that could explain the error

[Akash1134]

Thank you for asking that question. Once you are approved of the waitlist, you should be access the listed models 🙂

[001123]

Awesome! I hope I can be accepted to the waitlist. 🙏

[001123]

Yay, I accepted, and I am very excited to try it.

[machar94]

@Akash1134 , unless stated otherwise, can we assume that the models are all FP32? No quantization? I specifically curious about Meta-Llama-3-70B-Instruct.

[emimrulkayes]

I am on the waiting list. I don't know how long I will wait here? But I am very excited to test it.

[silverkiwi]

Also on waitlist ... avg wait time I heard was a few hours if you're successful, otherwise it's like waiting for Sora.

[Kishlay-notabot]

I've been accepted! happened a few days ago... So I don't think it would take too long.

[emimrulkayes]

I've been accepted! happened a few days ago... So I don't think it would take too long.

Congratulations 🎊

[7086cmd]

I submitted the application for it approximately eight weeks ago, but it has not yet been accepted. Could you please provide me with an update on the status of my application?

[emimrulkayes]

Yeah! finally I've been accepted! happened a few days ago.

[khairu-aqsara]

I have tested several models with the Continue Dev VsCode Plugin, and everything seems to be working well.

[prime399]

how did you configured, I don't see any option for using github model's API in continue plugin?

[maoling1411]

Everything is great, but the TPD and the 8192 input limit are really too low for me as a heavy AI user.

[RDjarbeng]

Still waiting after a month, to get off the waitlist

[yashksaini-coder]

I always wanted to build an application using Open AI but it required API keys, for which I don't have the means to purchase. but finally with Github, I can access various models for completely free.

Alright can anyone suggests some basic ideas to build for a complete beginner to practice once we get the access for the Github Models ?

[JuneReyP]

Hi, you can actually use the free tier for open AI

[yashksaini-coder]

Wow Thanks for this how do i get that @JuneReyP

[Inglan2]

I think its like a free trial

[mendhak]

Could you/anyone share some use cases for this, or something they're envisioning?

I understand the playground aspect, I've been experimenting with a few of the models. I like that the github token can be used here which makes it easy to call from a Github Action, for example.

[BaseMax]

Good as a start: https://www.youtube.com/watch?v=WiBB8Lsgl7I

[FaheemOnHub]

Guys as we have gained access to these models, how can we fine-tune or prompt engineer them or add retrieval-augmented generation ?

[rajbos]

@FaheemOnHub , have you looked at the samples in the Codespace? See a RAG example here: https://github.com/github/codespaces-models/blob/main/cookbooks/python/llamaindex/rag_getting_started.ipynb

Finetuning is not available as the offering to GitHub Models does not store any index or user data to fine tune with. That's also more compute intensive and therefore production like in my opinion. Perhaps that it is added in the future, who knows?

[KateCatlin]

Hey @FaheemOnHub! I'd love to chat with you about your use case for prompt engineering, we're planning some new tooling around that and I would love to know if it works for you. Can we set up time to chat? If so, you can book me here!

[azman0101]

I receive a mail "You've been granted access to the limited public preview of OpenAI o1 models on GitHub!"
But when I trying to use it, I'm asked to join the waitlist.

[dnafication]

What are the conditions for using these marketplace models? I'm aware that they are intended for trial purposes and not for production use. Could you provide information on the privacy policies? Is it permissible to test these models with private data? Also, do you monitor or have access to the usage of these models?

[dnafication]

Just found them:

Subject to Product Terms & Privacy Statement.

[KateCatlin]

Hey @dnafication, I'd love to learn more about your use case here and what private data you were hoping to test these models with. We have some upcoming releases I think you'd have great feedback about. If you're open to having a call with us, you can book me here!

[martasp]

Allow access to these models from the CI CD workflow environment when using ${{ secrets.GITHUB_TOKEN }} in workflows.

Workflow example:
...
permissions:
contents: read
AIModels: read
...

[KateCatlin]

Hey @martasp this is a really interesting suggestion and I'd love to learn more. Are you open to having a call with us to tell us what you'd want to do with that? If so, you can book me here!

[razvanphp]

llama 3.2 vision instruct does not work for me.

[KateCatlin]

Hey @razvanphp can you tell us more? Are the other models working? Are you using a VPN?

[huanphung12345]

In the github Models section, you don't allow us to use Vietnamese ip, so please let me use ip vietnam I have to fake ip to use that.

[matt-buchanan]

The documentation isn't clear on this, so thought I'd ask here - is there a way to control access to GitHub Models? It feels like there's a missing option in the Copilot policy set. I've got an Enterprise Cloud/EMU instance, keen to ensure that we have some oversight and control over the usage of preview features.

[rajbos]

It's not part of Copilot but of Models, so check the Policy for that. There you can enable/disable the functionality, but not a specific model itself.

[matt-buchanan]

Ah, OK, it's an enterprise control, thanks for flagging that - I had an expectation that we could shut off access to Models entirely but with the control disabled the UI is available but it makes any query just error out - it's maybe not the best user experience but it gives the desired outcome.

Models appears to be enabled via Copilot from what I can tell from the documentation - all the rate limits are based on Copilot tiers (https://docs.github.com/en/github-models/prototyping-with-ai-models#rate-limits) hence my assumption that it would be governed by Copilot policy.

[KateCatlin]

Hi @matt-buchanan! Glad you posted. Controlling whether GitHub Models is on and what models your team has access to is a feature we're currently building. I would love to learn more about your use case and whether our designs will work for you. If you have time for a 30-minute call, I'd love to connect!

[huanphung12345]

Ok, thanks for the clarification.

…

On Tue, Feb 11, 2025, 16:53 matt-buchanan ***@***.***> wrote: Ah, OK, it's an enterprise control, thanks for flagging that - I had an expectation that we could shut off access to Models entirely but with the control disabled the UI is available but it makes any query just error out - it's maybe not the best user experience but it gives the desired outcome. Models appears to be enabled via Copilot from what I can tell from the documentation - all the rate limits are based on Copilot tiers ( https://docs.github.com/en/github-models/prototyping-with-ai-models#rate-limits) hence my assumption that it would be governed by Copilot policy. — Reply to this email directly, view it on GitHub <#134377 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A6XNEMAOAVUCWP4WMBPVSML2PHCAJAVCNFSM6AAAAABL327WLSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMJTG44DIMY> . You are receiving this because you commented.Message ID: ***@***.***>

[huanphung12345]

cảm ơn bạn đã sửa lỗi, tôi rất vui khi bạn đã lắng nghe, cảm ơn bạn rất nhiều, thank you very much

…

On Tue, Feb 11, 2025, 18:38 Phùng Huấn ***@***.***> wrote: Ok, thanks for the clarification. On Tue, Feb 11, 2025, 16:53 matt-buchanan ***@***.***> wrote: > Ah, OK, it's an enterprise control, thanks for flagging that - I had an > expectation that we could shut off access to Models entirely but with the > control disabled the UI is available but it makes any query just error out > - it's maybe not the best user experience but it gives the desired outcome. > > Models appears to be enabled via Copilot from what I can tell from the > documentation - all the rate limits are based on Copilot tiers ( > https://docs.github.com/en/github-models/prototyping-with-ai-models#rate-limits) > hence my assumption that it would be governed by Copilot policy. > > — > Reply to this email directly, view it on GitHub > <#134377 (reply in thread)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/A6XNEMAOAVUCWP4WMBPVSML2PHCAJAVCNFSM6AAAAABL327WLSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMJTG44DIMY> > . > You are receiving this because you commented.Message ID: > ***@***.*** > com> >

[huanphung12345]

Yes, I have time.

…

On Tue, Feb 11, 2025, 23:46 Kate Catlin ***@***.***> wrote: Hi @matt-buchanan <https://github.com/matt-buchanan>! Glad you posted. Controlling whether GitHub Models is on and what models your team has access to is a feature we're currently building. I would love to learn more about your use case and whether our designs will work for you. If you have time for a 30-minute call, I'd love to connect <https://calendar.google.com/calendar/u/0/appointments/schedules/AcZssZ2n3vu9veZadjjWXpH6zCME6xnYQbZETKtrp2GnAFU3eAfQ6jlNQeq4c-3LqCqVjGzUaxl64fn0> ! — Reply to this email directly, view it on GitHub <#134377 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A6XNEMBSUB3LZYYT7RUZMGT2PISNTAVCNFSM6AAAAABL327WLSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMJUG4YDEMI> . You are receiving this because you commented.Message ID: ***@***.***>

[huanphung12345]

tôi rất hài lòng về github models nó đã cho tôi nhiều hiểu sâu về mọi thứ

…

On Wed, Feb 12, 2025, 08:41 Phùng Huấn ***@***.***> wrote: Yes, I have time. On Tue, Feb 11, 2025, 23:46 Kate Catlin ***@***.***> wrote: > Hi @matt-buchanan <https://github.com/matt-buchanan>! Glad you posted. > Controlling whether GitHub Models is on and what models your team has > access to is a feature we're currently building. I would love to learn more > about your use case and whether our designs will work for you. If you > have time for a 30-minute call, I'd love to connect > <https://calendar.google.com/calendar/u/0/appointments/schedules/AcZssZ2n3vu9veZadjjWXpH6zCME6xnYQbZETKtrp2GnAFU3eAfQ6jlNQeq4c-3LqCqVjGzUaxl64fn0> > ! > > — > Reply to this email directly, view it on GitHub > <#134377 (reply in thread)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/A6XNEMBSUB3LZYYT7RUZMGT2PISNTAVCNFSM6AAAAABL327WLSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMJUG4YDEMI> > . > You are receiving this because you commented.Message ID: > ***@***.*** > com> >

[huanphung12345]

In the github Models section, you don't allow us to use Vietnamese ip, so please let me use ip vietnam I have to fake ip to use that.

[chuyangliu]

Is there a dashboard that I can look at the number of requests sent for each model per day?

[huanphung12345]

Yes, I do. Vào Th 7, 8 thg 3, 2025 vào lúc 07:53 chuyangliu ***@***.***> đã viết:

…

Is there a dashboard that I can look at the number of requests sent for each model per day? — Reply to this email directly, view it on GitHub <#134377 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A6XNEMFBP457OZUAZITY5632TI5Q3AVCNFSM6AAAAABL327WLSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTENBTGE4DSNI> . You are receiving this because you commented.Message ID: ***@***.***>