[Feature Request] Nightly or Weekly Android apk build

[EwoutH]

🚀 Feature

Currently and Android APK is provided on https://llm.mlc.ai/docs/deploy/android.html. However, it's already a few months old and not updated on a regular schedule.

It would be great to have a nightly or weekly build of the Android APK, available on an accessible place

If needed, there could be separate versions for Adreno and Mali GPUs.

Motivation

With a nightly or weekly build, users can always grab the latest version and know when a new version will be build after a feature merge.

[tqchen]

Thanks for th suggestion, deploying to mobile would involve more checks, so likely we are going to follow a release schedule in future and not doing nightly for apks

[EwoutH]

A (predictable) release schedule would also be great. For example, it looks like the Android APK was updated 3 days ago, but there is no version number or changelog available. This makes difficult to see if certain functionality is supported or not.

[sebastienbo]

I agree with Ewouth.
You guys have found a hole in the market, you guys have the only android application that can run LLM's on a phone.
I use phi-2, and wow this is great. Phi-3 just came out and that one is even much better then Llama 2 70b in just 2GB of data (incredible if this could work on a phone , especially now that snapdragon has a NPU onboard to offload LLM's -> much faster token generation and consumes less battery)

[tqchen]

we are movig towards a JIT based SDK system in https://llm.mlc.ai/docs/deploy/android.html which hopefully allows latest features to be incorporated by directly re-runing the mlc_llm package.

We likely wont do nightly apk build in near term but hope the SDK plus a apk release after the Engine migration can help people to build apps around, please also stay tuned for future MLCEngine updates #2217 .

[sebastienbo]

Hi, The problem is not only rebuilding it, but also host the apk for others. I'm afraid it will take moths before we will see a custom mlc app again :-(

…

On Sat, 11 May 2024, 05:11 Tianqi Chen, ***@***.***> wrote: we are movig towards a JIT based SDK system in https://llm.mlc.ai/docs/deploy/android.html which hopefully allows latest features to be incorporated by directly re-runing the mlc_llm package. We likely wont do apk build but hope the SDK plus a release after the Engine migration can help people to build apps around, please also stay tuned for future MLCEngine updates #2217 <#2217> . — Reply to this email directly, view it on GitHub <#2194 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB7MPQV7XHWPC3Z2U4PV5TLZBWD5NAVCNFSM6AAAAABGS5I5T2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBVGUYDIMRWGY> . You are receiving this because you commented.Message ID: ***@***.***>

[tqchen]

@sebastienbo sorry just wan to clarify, we would like to host APK for stable releases :) e.g. with major models, i just mention that in near term it would be harder for nightly

[sebastienbo]

@sebastienbo sorry just wan to clarify, we would like to host APK for stable releases :) e.g. with major models, i just mention that in near term it would be harder for nightly

Cool thank you

When do you expect a APK with phi 3?
How can we recognise it? Because your current APK hosted on GitHub does not have a version name in its file name

Ps: why is it necessary to download a new apk for each new model? Wouldn't it be easier to have an externally updated list? It would update the internal list. That way you can add new models just by adding them to the externally hosted list. And the user would just press a refresh button to see all the supported models (the list) that would also solve the problem of accidently deleting a model