{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":780319624,"defaultBranch":"main","name":"web-llm","ownerLogin":"jzhao62","currentUserCanPush":false,"isFork":true,"isEmpty":false,"createdAt":"2024-04-01T08:07:59.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/25937657?v=4","public":true,"private":false,"isOrgOwned":false},"refInfo":{"name":"","listCacheKey":"v0:1711958885.2672482","currentOid":""},"activityList":{"items":[{"before":"c6998f29640fd86689fc64b103b92faecb292a78","after":"fe89f36604809eb40f57f6cbf62229bf87725c99","ref":"refs/heads/main","pushedAt":"2024-05-22T10:08:25.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"pull[bot]","name":null,"path":"/apps/pull","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/12910?s=80&v=4"},"commit":{"message":"[Version] Bump version to 0.2.37 (#409)\n\nThis version bump is breaking as we renamed APIs.\r\n\r\n### Changes\r\nThe only two changes are:\r\n- Renamed all `Engine` to `MLCEngine`\r\n - https://github.com/mlc-ai/web-llm/pull/408\r\n- See\r\nhttps://github.com/mlc-ai/web-llm/pull/408/files#diff-a2a171449d862fe29692ce031981047d7ab755ae7f84c707aef80701b3ea0c80\r\nfo the full list of changes\r\n- To accommodate this breaking change, control-F `Engine` (with same\r\ncapitalization) and replace with `MLCEngine` should suffice\r\n- Renamed ServiceWorker classes:\r\n - https://github.com/mlc-ai/web-llm/pull/403\r\n - WebServiceWorker -> ServiceWorker\r\n - ServiceWorker -> ExtensionServiceWorker\r\n\r\n### WASM Version\r\nv0_2_34 as no change is required.\r\n\r\n### TVMjs\r\nTVMjs compiled at\r\nhttps://github.com/apache/tvm/commit/a5862a5c696a3237f644f31bc312aae303213f3f,\r\nwith no change","shortMessageHtmlLink":"[Version] Bump version to 0.2.37 (mlc-ai#409)"}},{"before":"8f91d01dcbe2f0580cc7c2f46a420a021627c111","after":"c6998f29640fd86689fc64b103b92faecb292a78","ref":"refs/heads/main","pushedAt":"2024-05-22T04:55:43.000Z","pushType":"push","commitsCount":5,"pusher":{"login":"pull[bot]","name":null,"path":"/apps/pull","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/12910?s=80&v=4"},"commit":{"message":"[Rename] Rename service worker export classes (#403)\n\nThis PR renames the following classes in export to better align with\r\ntheir well-known names.\r\n\r\n- WebServiceWorker -> ServiceWorker\r\n https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API\r\n- ServiceWorker -> ExtensionServiceWorker\r\n\r\nhttps://developer.chrome.com/docs/extensions/develop/concepts/service-workers\r\n\r\nNote, this is a **breaking** change for existing extension service\r\nworker engine users. Therefore, please merge with caution.","shortMessageHtmlLink":"[Rename] Rename service worker export classes (mlc-ai#403)"}},{"before":"dd5ec5c25bb7c8524f65a47da16a8b45079c5dd9","after":"8f91d01dcbe2f0580cc7c2f46a420a021627c111","ref":"refs/heads/main","pushedAt":"2024-05-21T08:56:19.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"pull[bot]","name":null,"path":"/apps/pull","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/12910?s=80&v=4"},"commit":{"message":"chore: update types.ts (#404)\n\noveride -> override","shortMessageHtmlLink":"chore: update types.ts (mlc-ai#404)"}},{"before":"ad04a7d3238836ee51303d5680b73be07632c721","after":"dd5ec5c25bb7c8524f65a47da16a8b45079c5dd9","ref":"refs/heads/main","pushedAt":"2024-05-19T22:05:53.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"pull[bot]","name":null,"path":"/apps/pull","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/12910?s=80&v=4"},"commit":{"message":"[Feature] Replace BroadcastChannel API with Client API in Service Worker (#401)\n\n## Overview\r\n\r\nThere are many different APIs available for the two-way communication\r\nbetween service worker and web pages. According to the suggestion of\r\nweb.dev, previously we used the simple BroadcastChannel API.\r\nhttps://web.dev/articles/two-way-communication-guide\r\n\r\nHowever, if the service worker has been killed by the browser, messages\r\nsent via the BroadcastChannel API cannot revive it. This causes unstable\r\nservice worker life while users are using the webapp.\r\n\r\nThis PR replaces BroadcastChannel API with the more primitive Client API\r\nas we found that messages sent via Client API will revive a stopped\r\nservice worker. This ensures the normal functioning of our keep-alive\r\nmechanism.\r\n\r\n## Primary Change\r\n\r\n- service_worker.ts:\r\n- Replace `BroadcastChannel` with Client API\r\n(`navigator.serviceWorker.controller.postMessage()` and\r\n`client.postMessage()`)\r\n- Add `clientRegistry` to remember mapping between incoming messages and\r\nclient ids\r\n - Add \r\n- Rename files (the export names are kept the same):\r\n - `web_service_worker.ts` -> `service_worker.ts`\r\n - `service_worker.ts` -> `extension_service_worker.ts`\r\n\r\n## Testing\r\n\r\n- https://chat.webllm.ai/\r\n- `examples/service-worker`\r\n\r\nThe chat webapp is able to correctly keeping service worker alive after\r\nthis change.","shortMessageHtmlLink":"[Feature] Replace BroadcastChannel API with Client API in Service Wor…"}},{"before":"4d1915efe3579f9cac7f29dfb77ad2a6025503e7","after":"ad04a7d3238836ee51303d5680b73be07632c721","ref":"refs/heads/main","pushedAt":"2024-05-18T19:07:15.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"pull[bot]","name":null,"path":"/apps/pull","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/12910?s=80&v=4"},"commit":{"message":"[Function] Add heartbeat to service worker (#400)\n\n## Overview\r\nThis PR adds heartbeat event in web service worker so that the client\r\ncan monitor its status and respond correspondingly.\r\n\r\n## Primary Changes\r\n- Update `{ type: \"keepAlive\" }` event to `{ kind: \"keepAlive\" }` to\r\nkeep all event format consistent\r\n- Add heartbeat event in web service worker handler to report back its\r\nstatus\r\n\r\n## Testing\r\n\"Screenshot","shortMessageHtmlLink":"[Function] Add heartbeat to service worker (mlc-ai#400)"}},{"before":"3379591964a1aaa546c335f9d072aa0b0df08d52","after":"4d1915efe3579f9cac7f29dfb77ad2a6025503e7","ref":"refs/heads/main","pushedAt":"2024-05-15T16:56:36.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"pull[bot]","name":null,"path":"/apps/pull","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/12910?s=80&v=4"},"commit":{"message":"[Fix] Avoid unnecessary engine reload by correctly comparing ChatOption and AppConfig objects (#399)\n\n## Overview\r\nCurrently, even the client is initializing the worker engine with the\r\nexact same configurations, the worker will not correctly recognize this\r\nbut instead it will unnecessarily re-initialize itself. The root cause\r\nis due to the use of `===` to compare object equity which is actually\r\ncomparing object reference equity instead of value equity.\r\n\r\nThis PR fixed it by create utility functions for deep comparing the\r\n**VALUE** of these config objects. The code is tedious and thus I\r\ngenerated using AI models.\r\n\r\n## Test\r\nTested on https://chat.neet.coffee with the following code added to\r\n`web_service_worker.ts`.\r\n\r\n```typescript\r\n console.log(\"modelId same? \" + this.modelId === params.modelId);\r\n console.log(\"chatOpts same? \" + areChatOptionsEqual(this.chatOpts, params.chatOpts));\r\n console.log(\"appConfig same? \" + areAppConfigsEqual(this.appConfig, params.appConfig));\r\n```\r\n\r\nBefore the fix:\r\n```\r\nmodelId same? true\r\nchatOpts same? false\r\nappConfig same? false\r\n```\r\n\r\nAfter:\r\n```\r\nmodelId same? true\r\nchatOpts same? true\r\nappConfig same? true\r\nAlready loaded the model. Skip loading\r\n```","shortMessageHtmlLink":"[Fix] Avoid unnecessary engine reload by correctly comparing ChatOpti…"}},{"before":"ce725b6422a55d4125c7ed05d729e17a5e65aff9","after":"3379591964a1aaa546c335f9d072aa0b0df08d52","ref":"refs/heads/main","pushedAt":"2024-05-14T21:52:29.000Z","pushType":"push","commitsCount":4,"pusher":{"login":"pull[bot]","name":null,"path":"/apps/pull","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/12910?s=80&v=4"},"commit":{"message":"docs: typo (#379)","shortMessageHtmlLink":"docs: typo (mlc-ai#379)"}},{"before":"87b58ed64904a5ac32cb9757a9c981279bfc039d","after":"ce725b6422a55d4125c7ed05d729e17a5e65aff9","ref":"refs/heads/main","pushedAt":"2024-04-23T03:02:49.000Z","pushType":"push","commitsCount":5,"pusher":{"login":"pull[bot]","name":null,"path":"/apps/pull","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/12910?s=80&v=4"},"commit":{"message":"[Version] Bump version to 0.2.35 (#378)\n\n### Changes\r\nVery minor change from 0.2.34, only slightly updated prebuilt config\r\n- https://github.com/mlc-ai/web-llm/pull/377\r\n\r\n### WASM Version\r\n`v0_2_34` as no change is required.\r\n\r\n### TVMjs\r\nTVMjs compiled at\r\nhttps://github.com/apache/tvm/commit/57316dae1497c36ed57732a7a610018a990f1927","shortMessageHtmlLink":"[Version] Bump version to 0.2.35 (mlc-ai#378)"}},{"before":"9949fe21cd1001a903e30b9751e6e10ad5aa3d42","after":"87b58ed64904a5ac32cb9757a9c981279bfc039d","ref":"refs/heads/main","pushedAt":"2024-04-20T07:00:34.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"pull[bot]","name":null,"path":"/apps/pull","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/12910?s=80&v=4"},"commit":{"message":"[SimpleChat] Treat user input as text rather than HTML (#370)\n\nInstead of putting users' input as arguments to `insertAdjacentHTML()`,\r\nwe use `insertAdjacentText` instead by creating a dedicated\r\n`appendUserMessage()` method.","shortMessageHtmlLink":"[SimpleChat] Treat user input as text rather than HTML (mlc-ai#370)"}},{"before":"bd2409c37af793bc7bc51e49ccec40553976d136","after":"9949fe21cd1001a903e30b9751e6e10ad5aa3d42","ref":"refs/heads/main","pushedAt":"2024-04-19T13:12:12.000Z","pushType":"push","commitsCount":7,"pusher":{"login":"pull[bot]","name":null,"path":"/apps/pull","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/12910?s=80&v=4"},"commit":{"message":"Update system prompts in examples","shortMessageHtmlLink":"Update system prompts in examples"}},{"before":"3c03d4c2cc78a8b24414f6afdafe82dbea297a88","after":"bd2409c37af793bc7bc51e49ccec40553976d136","ref":"refs/heads/main","pushedAt":"2024-04-16T04:14:29.000Z","pushType":"push","commitsCount":2,"pusher":{"login":"pull[bot]","name":null,"path":"/apps/pull","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/12910?s=80&v=4"},"commit":{"message":"[Version] Bump version to 0.2.32 (#365)\n\n## Changes\r\n- Add ServiceWorkerEngine support. Migrate chrome extension example to\r\nuse ServiceWorkerEngine.\r\n - https://github.com/mlc-ai/web-llm/pull/364\r\n\r\n## WASM Version\r\nWASMs are still v0_2_30 as no compile-time changes are required. \r\n\r\n## tvmjs\r\nhttps://github.com/mlc-ai/relax/commit/c7bdcabd602f3d882e764232692d1d1eb449d07b\r\ni.e.\r\nhttps://github.com/apache/tvm/commit/d1e24ca721d0c8110fab3e7db6b7375ebfeb8ac5","shortMessageHtmlLink":"[Version] Bump version to 0.2.32 (mlc-ai#365)"}},{"before":"5e3c1ed71a7c2f88655617f9202d71a8277ad662","after":"3c03d4c2cc78a8b24414f6afdafe82dbea297a88","ref":"refs/heads/main","pushedAt":"2024-04-15T09:57:12.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"pull[bot]","name":null,"path":"/apps/pull","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/12910?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"f6ef70a9ec526a9ce19e88b0797482609843cc06","after":"5e3c1ed71a7c2f88655617f9202d71a8277ad662","ref":"refs/heads/main","pushedAt":"2024-04-10T04:08:56.000Z","pushType":"push","commitsCount":3,"pusher":{"login":"pull[bot]","name":null,"path":"/apps/pull","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/12910?s=80&v=4"},"commit":{"message":"[Version] Bump version to 0.2.31 (#362)\n\n### Changes\r\n- Enable `webllm.AppConfig.useIndexedDBCache` so that user can choose\r\nbetween the Cache API and IndexedDB Cache\r\n - https://github.com/mlc-ai/web-llm/pull/352\r\n- Refactor `ChatModule` to `Engine`, finalize OpenAI API to\r\n`engine.chat.completions.create()` to match that of OpenAI\r\n - https://github.com/mlc-ai/web-llm/pull/361\r\n- Add deprecation warning of `engine.generate()`, suggesting user to use\r\n`engine.chat.completions.create()`\r\n- Subsequently, support implicit KVCache reuse for multi-round chat\r\nusage\r\n- Though `chat.completions.create()` is functional, hence requiring\r\nusers to maintain all chat history, we will reuse KVs if we detect\r\nmulti-round chat, hence not sacrificing any performance\r\n- See https://github.com/mlc-ai/web-llm/pull/359 and\r\n`examples/multi-round-chat`\r\n\r\n### WASM Version\r\n\r\nWASMs are still `v0_2_30` as no compile-time changes are required.\r\n\r\n### tvmjs\r\nUpdated tvmjs to:\r\n-\r\nhttps://github.com/mlc-ai/relax/commit/c7bdcabd602f3d882e764232692d1d1eb449d07b\r\n- i.e.\r\nhttps://github.com/apache/tvm/commit/d1e24ca721d0c8110fab3e7db6b7375ebfeb8ac5","shortMessageHtmlLink":"[Version] Bump version to 0.2.31 (mlc-ai#362)"}},{"before":"0d003f8595511aa9c7320cf591ce22985cb18f39","after":"f6ef70a9ec526a9ce19e88b0797482609843cc06","ref":"refs/heads/main","pushedAt":"2024-04-06T01:15:18.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"pull[bot]","name":null,"path":"/apps/pull","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/12910?s=80&v=4"},"commit":{"message":"[Examples] Update model link to reflect prebuilt changes (#360)\n\nSince we exposed all prebuilt links to `prebuiltAppConfig`, we update\r\nhow examples select models correspondingly.","shortMessageHtmlLink":"[Examples] Update model link to reflect prebuilt changes (mlc-ai#360)"}},{"before":"d20e08c1841d95b2211de5b2302092d062d121a6","after":"0d003f8595511aa9c7320cf591ce22985cb18f39","ref":"refs/heads/main","pushedAt":"2024-04-04T20:05:50.000Z","pushType":"push","commitsCount":4,"pusher":{"login":"pull[bot]","name":null,"path":"/apps/pull","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/12910?s=80&v=4"},"commit":{"message":"[KVCache] Add implicit KVCache reuse, disable stateful option for chatCompletion (#359)\n\nWe introduced the field `stateful` in `chatCompletion()` earlier to\r\nallow easier multi-round chatting in\r\nhttps://github.com/mlc-ai/web-llm/pull/330.\r\n\r\nHowever, this is not ideal since we would prefer APIs that are\r\nfunctional in behavior, giving us various benefits (e.g. better fault\r\ntolerance for future use cases).\r\n\r\nTherefore, in this PR:\r\n- We disable `chatCompletionRequest.stateful`, and ask users to maintain\r\nthe chat history explicitly\r\n- Instead, we introduce implicit KVCache reuse for multi-round chatting\r\n- When we detect users are doing multi-round chatting, we will not reset\r\nthe KV cache, so only the new message will be prefilled\r\n- To detect multi-round chatting, we instantiate a `Conversation`\r\ninstance for each request, and compare it with the current internal\r\n`Conversation`. If they match, it means that we can safely not reset the\r\ninternal state, and only prefill the new input.\r\n\r\nTo see the behavior, check out `mainMultiroundChat()` in\r\n`examples/openai-api/src/openai_api.ts`.\r\n\r\nImplementation details:\r\n- Instantiate `Conversation` object in `ChatModule.prefill()`, since\r\nthis is the place where various workflows meet (streaming,\r\nnon-streaming, n > 1, etc.)\r\n- The object's state is determined by system prompt, message history,\r\nand function calling usages\r\n- Inside `prefill()`, we then compare the two objects with\r\n`compareConversationObject()`, reset all internal states if false\r\n- Another detail is that, instead of overriding\r\n`conversation.config.system_message`, we add a field\r\n`conversation.override_system_message`, making `conversation.config`\r\nprotected\r\n- We further remove all methods in `ChatModule` that overrides\r\n`this.getPipeline().conversation` by changing\r\n`updateConversationWithChatCompletionMessages()` to\r\n`getConversationFromChatCompletionRequest()`, keeping things more\r\nfunctional internally","shortMessageHtmlLink":"[KVCache] Add implicit KVCache reuse, disable stateful option for cha…"}},{"before":"e4e2903c68930bb71b47519d4418e81c78eb9ccb","after":"d20e08c1841d95b2211de5b2302092d062d121a6","ref":"refs/heads/main","pushedAt":"2024-04-02T08:34:06.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"pull[bot]","name":null,"path":"/apps/pull","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/12910?s=80&v=4"},"commit":{"message":"Allow setting top_p to 1.0 (#354)","shortMessageHtmlLink":"Allow setting top_p to 1.0 (mlc-ai#354)"}}],"hasNextPage":false,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"Y3Vyc29yOnYyOpK7MjAyNC0wNS0yMlQxMDowODoyNS4wMDAwMDBazwAAAARQxm6n","startCursor":"Y3Vyc29yOnYyOpK7MjAyNC0wNS0yMlQxMDowODoyNS4wMDAwMDBazwAAAARQxm6n","endCursor":"Y3Vyc29yOnYyOpK7MjAyNC0wNC0wMlQwODozNDowNi4wMDAwMDBazwAAAAQlR31y"}},"title":"Activity · jzhao62/web-llm"}