Move and refactor EmbeddedContent/Store and clean up core services #19
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cbush
changed the title
mongodben
reviewed
Jun 30, 2023
| @@ -0,0 +1,90 @@ | |||
| import { PersistedPage } from "./Page"; | |||
|
|
|||
| export interface EmbeddedContent { | |||
There was a problem hiding this comment.
i think we're also adding score per https://github.com/mongodb/docs-chatbot/pull/19/files#diff-e4eb4230c061137b41887df64b518d22d061c49648bc27b90621b206dc08faadR115
There was a problem hiding this comment.
Done (but not in the actual EmbeddedContent type) --> 7ebd62f
mongodben
reviewed
Jun 30, 2023
| /** | ||
| Creates an OpenAI implementation of the embedding function. | ||
| */ | ||
| export const makeOpenAiEmbedFunc = ({ |
There was a problem hiding this comment.
we should refactor this to use https://www.npmjs.com/package/@azure/openai
but we can save that for future work. all internal changes, so the API that we're adding here would stay same.
There was a problem hiding this comment.
(actually input params might change..but we can deal with that later)
mongodben
reviewed
Jun 30, 2023
chat-core/src/assertEnvVars.ts
Outdated
mongodben
reviewed
Jun 30, 2023
| OpenAiEmbeddingsClient, | ||
| OpenAiChatClient, | ||
| } from "../../src/integrations/openai"; | ||
| import { OpenAiChatClient } from "../../src/integrations/openai"; |
There was a problem hiding this comment.
why are we just removing embedding stuff from openai integration but not also llm?
perhaps future work
There was a problem hiding this comment.
Yeah, as big as this refactor is, the actual purpose is just to allow ingest to use embed. I left as much of the other stuff alone as I could.
mongodben
reviewed
Jun 30, 2023
chat-server/src/app.ts
Outdated
| } = assertEnvVars(CORE_ENV_VARS); | ||
|
|
||
| // Create instances of services | ||
| const mongodb = new MongoDB( |
There was a problem hiding this comment.
in future PR we're killing this MongoDB as a self defined type to just use the DB
mongodben
reviewed
Jun 30, 2023
mongodben
reviewed
Jun 30, 2023
chat-server/src/app.ts
Outdated
Comment on lines
96
to
100
| return Object.assign(app, { | ||
| async close() { | ||
| await mongodb.close(); | ||
| await store.close(); | ||
| }, |
There was a problem hiding this comment.
why separate mongodb and store? aren't using the same DB?
There was a problem hiding this comment.
This should be refactored but refactoring ConversationsService etc. was not in scope
mongodben
approved these changes
Jun 30, 2023
mongodben
reviewed
Jun 30, 2023
chat-server/src/app.ts
Outdated
| @@ -71,7 +37,41 @@ const reqHandler: RequestHandler = (req, _res, next) => { | |||
| next(); | |||
| }; | |||
|
|
|||
| export const setupApp = async () => { | |||
| export const makeApp = async (): Promise< | |||
There was a problem hiding this comment.
to make the app more testable, i think it'd make sense to pass mongodb, conversationService, store, dataStreamer, and embed as args to makeApp().
then we can pass these things in in the index.ts function which is the only file that's actually involved in running the app.
WIP: Implement embedder Significantly clean up the EmbedFunc implementation Apply changes to server Implement embedding content Move DatabaseConnection to core library Use chat-core objects for ingest tool
Co-authored-by: Ben Perlmutter <90647379+mongodben@users.noreply.github.com>
mongodben
added a commit
that referenced
this pull request
Nov 2, 2023
* update lock (e45350d) * update core version (7d15d35) * update release conf (05a6686) * remove unused dep (12c73a3) * remove test from release-it b/c covered in CI (99f2a9b) * update drone (932c9a8) * Release mongodb-rag-ingest v0.1.6 (d9e3ea9) * update lock (bd718e6) * update lock (dac1773) * update dependent packages (50a43db) * Release mongodb-rag-core v0.0.5 (55195cb) * Release mongodb-rag-core v0.0.5-3 (b765423) * remove unused bundled dep (87f6f7e) * make ingest reexport core (5b0fd8e) * Add missing typechat dep (4bf2594) * Release mongodb-rag-ingest v0.1.6-2 (a22fbdf) * update lock file (def3cb7) * bunlde chat-core in ingest (3f763fe) * Release mongodb-rag-ingest v0.1.6-1 (fdd536c) * fix drone file (9b16f20) * Release mongodb-rag-ingest v0.1.6-0 (767efac) * update drone publication pipelines (3d761a6) * Release mongodb-rag-core v0.0.5-2 (a5689a2) * fix drone script (f711595) * Release mongodb-rag-core v0.0.5-1 (0a3745a) * Release mongodb-rag-core v0.0.5-0 (580e8ff) * update npm chat-core publish config (5be4922) * update chatbot ui out destination (d3e20a2) * fix build err from test (af0cce0) * update chat-core dependency (069fb9b) * Trigger staging build of mongodb server (a438a82) * update chat-server.dockerfile (81999da) * update executable path (5b06c12) * update mongodb ingest to use latest ingest CLI (b21441f) * Release mongodb-rag-ingest v0.1.5 (bf852d1) * update lock (70a8c65) * make CLI entry point executable (00b57d3) * trigger build of ingest-mongodb-public to test drone (96b4206) * update package-lock (ff83848) * update package-lock.json w new deps (f4d6579) * update dependencies to latest in mongodb ingest (0d6e48a) * Release mongodb-rag-ingest v0.1.4 (7b7e8de) * update monorepo chatcore version (80ebd61) * update ingest version of core to latest working (5a7b346) * Release mongodb-rag-core v0.0.4 (e04b754) * Add simple readme to core (729f44a) * Release mongodb-rag-core v0.0.3 (230ab97) * update package.json files with files to include in release (e4567ed) * Release mongodb-rag-core v0.0.2 (e5e6ac1) * make ingest a node executable (da4011f) * simplify dockerfile (e2d8751) * fix build commands (37d746b) * retry drone (bbbdd82) * trigger ingest build again (1315732) * hack drone to trigger builds (88e3255) * trigger ingest staging build (33c7f34) * Release mongodb-rag-core v0.0.2-8 (2258731) * add publish env vars (1ed87ad) * Release mongodb-rag-core v0.0.2-7 (c5914d3) * fix build issues (965a9e0) * fix flaky tests (9dc7153) * fix broken server tests (e4b04b9) * make ingest tests pass (00c9065) * remove chatbot-server-mongodb-public tests from prod build (d95c984) * remove chat-server tests from prod build (4c0cd22) * working chat server (0248a11) * update core version in server (d236f91) * update package files to use (7d3e5b8) * Release mongodb-rag-core v0.0.2-6 (40688a7) * update pub flow to speed up testing (8e0409c) * add new npm creds (cf6084a) * update drone names (47dd13c) * remove release-it from monorepo conf (ff4434c) * separate chat server package and implementation + update drone file (4441cbf) * update core + ingest release conf (5559403) * Release mongodb-rag-ingest v0.1.3 (9a7a2f9) * add non prerelease version (756d1a7) * update deps (7f4ac98) * update ingest conf (88c47ee) * Release mongodb-rag-core v0.0.2-5 (ba238e6) * add .drone pipelines to deploy new npm packages (f4a8210) * configure so that publishes package from creation of gh tag via drone (f9df609) * Release mongodb-rag-core v0.0.2-4 (d626d91) * remove from / to from github (881e474) * update release conf (0477bc2) * inside (should see) (7f19587) * add outside commit (shouldn't see) (feccfe7) * inside (c22271b) * all (90b568a) * Release mongodb-rag-core v0.0.2-3 (a90c1ba) * Add changelog (4a9af69) * release notes/changelog (c9b3b0f) * add changelog setup (3e90095) * Release mongodb-rag-core v0.0.2-2 (bcf3c9c) * Release mongodb-rag-core v0.0.1 (a3a1355) * update version (f6e1d8f) * add release-it to chat-core (629de3e) * update release conf (73308a6) * update contributing guide (11ee87b) * update repo to use lerna versioning (f7c7285) * Merge remote-tracking branch 'upstream/main' into oss-refactor-pt-2 (412da6a) * Ingest plugins (#211) (0206bee) * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) (b8541a7) * OSS Chat Server (#206) (820e249) * (DOCSP-32603): Add Terraform data source (#205) (0d64743) * dev QoL scripts (3813ad7) * (DOCSP-33744): update QA server DB name for latest snapshot (#210) (5df2bf0) * (DOCSP-32482): Build out qualitative tests (#201) (0535a89) * Organize some files (#207) (f8cdd3c) * Conversations refactor (#199) (5a60a15) * Release ingest-v0.1.2 (#204) (ed34507) * [Snyk] Upgrade urllib from 3.19.1 to 3.19.2 (#203) (743cc01) * Add Rust and Entity Framework Snooty data sources (#202) (ff15139) * [Snyk] Upgrade urllib from 3.19.0 to 3.19.1 (#200) (dea05e0) * Release chat-server v0.1.3 (ac542c8) * Use GITHUB_TOKEN for chat-ui releases (38a9e1a) * [Snyk] Upgrade urllib from 3.18.1 to 3.19.0 (#197) (dd2baaf) * Do not fail query on rephrase fail (#196) (8b7c60a) * Release chat-ui v0.0.8 (03bf7cb) * Bundle new @lg-chat components (2f1f11b) * Release chat-ui v0.0.7 (d082a63) * Unify release-it config (419422e) * Release chat-server v0.1.2 (b0b425c) * [Snyk] Upgrade urllib from 3.18.0 to 3.18.1 (#194) (2c98ca4) * [DOCSP-32950] Create a DevCenter Chatbot Component (#193) (c219d03) * (DOCSP-33445) [ANALYTICS] Scrubbed Views (#195) (8ef2bd7) * (DOCSP-33419) [ANALYTICS] Operational Views (#192) (aae8259) * Release scripts v0.1.2 (fbf2bbf) * Add whether message was deemed "Do not answer" to conversation (#191) (955b722) * (DOCSP-31104) [UI] Add tests to FE services (#190) (d3e2000) * Release ingest v0.1.1 (5b06cd2) * Update package-lock.json (a568d00) * Replace GitHub data source (#188) (747d08e) * Update env vars for analyze script (0a46085) * Add scrubbed message analyzer (#187) (9cef6b8) * (DOCSP-33291) [UI] Add tck tracking to links (#185) (acfb6a1) * Update README.md (a38cbf7) * Release 0.1.1 (8ee27c1) * Fix typo in scripts build (0534458) * Release 0.1.0 (09fa262) * Add release-it to scripts (9ea2e71) * Trigger drone run for scripts service (b1ffcc8) * Fix drone scripts (#186) (9bae8e9) * Cron scrub (#184) (d67dda9) * (DOCSP-33273) Open Source Licensing (#183) (00611bf) * De-flake SnootyProjectsInfo test (#181) (9f7250e) * (DOCSP-32930): Add pymongo metadata (#179) (6694209) * Update release-it git add config to add monorepo stuff (679e447) * Update package-lock in monorepo for release (e7308ef) * Release 0.1.1 (d60d14b) * Update release-it pre-hooks (718b96d) * Run npm audit fix (#178) (c74e1a9) * Remove `seed-content` package (#157) (1b9ac43) * Add scrub message script (#177) (ee2f864) * Remove extra padding around initial input (9dc27ad) * (DOCSP-33094): Save conversation embeddings (#175) (9c6e79d) * Ingest release 0.1.0 (#174) (4928eed) * Release chat-ui-v0.0.6 (2bf2a0e) * (DOCSP-33059) [UI] Fix suggested prompts & other LG Chat tweaks (#171) (cde1b74) * chat server v0.1.0 (#170) (ba69ad8) * new commit for build (56ed01e) * trigger ingest build (4052af9) * clean up spec + remove ip address validation from rate msg (#166) (28cf222) * UI Tweaks (#159) (53b392f) * Delete chat-server/trigger.me (#165) (5542f41) * update repo name in code (#163) (f9628bb) * Update README.md (5ad9cd0) * Launch pre releases (#162) (9b2380b) * Expand Contributing Guide and `ingest` README (#161) (30434e3) * filter out very small chunks (#160) (d952c6d) * (DOCSP-31928): Implement numbered lists (#154) (f41258d) * clarify ingest readme (115b00b) * move git install to correct docker stage (40600cb) * (DOCSP-32991, DOCSP-32992): `GitDataSource`, functionality for converting HTML to Markdown, ingest Scala, Java Reactive Streams, and C docs (#152) (7118444) * (DOCSP-32915) [UI] Streamer bugs out when switching tabs (#156) (91c288d) * (DOCSP-33090) [UI] Remove experimental badge from input on focus (#155) (a3ea236) * (DOCSP-32993, DOCSP-32906, DOCSP-32993, DOCSP-32990): Data sources for MD on Github + data ingestion of related content (#148) (7daed63) * change recommended prompt (#150) (2521f08) * add missing env example (#153) (1201a34) * [Snyk] Upgrade mongodb from 5.8.0 to 5.8.1 (#151) (966a4bf) * make preprocessor less biased toward shell (#149) (886cbe5) * (DOCSP-32616): Refactor config, add library, and log config (#143) (4fcfc58) * add default + remove unused style (0296b42) * Update chat-ui release config (3174b61) * Release chat-ui-v0.0.5 (65727f7) * Add root release:chat-ui command (4621989) * (DOCSP-32757) [UI] Update to Responsive LG Chat Components (#145) (81b70bb) * (DOCSP-32670): Basic server load testing (#136) (282c947) * [Snyk] Upgrade urllib from 3.17.2 to 3.18.0 (#147) (aefa9e0) * (DOCSP-32756): Implement headings in tables (#140) (ea416ce) * custom/synthetic/meta data source content (#127) (d687312) * [Snyk] Upgrade urllib from 3.17.1 to 3.17.2 (#144) (209135a) * (DOCSP-32673) [UI] Suppress focus to prevent modal reopen (#139) (43b8a98) * (DOCSP-32936): Increase single IP rate limits to be more line w firewall rate limits (#141) (87d8785) * (DOCSP-31863): Fix flaky test, refactor core llm service + server implementation (#142) (584a5e3) * (DOCSP-32916): Conversation service rate limiting + slow down (#138) (a88a5d5) * (DOCSP-32914): Stop unindenting context (#137) (27630c6) * (DOCSP-31285): Ingest PyMongo docs (#118) (e04c10b) * (DOCSP-32694) Dark mode: use LG palette & provider + cleanup (a95f5c5) * (DOCSP-32694) [UI] Add darkMode prop (#129) (820229c) * (DOCSP-32896) Remove mention of <CONTEXT> to reduce damage of prompt leakage (#135) (1484426) * Update internal docs (#134) (a3b2f56) * Handle Snooty Data API tombstones (#133) (a775da4) * (DOCSP-32867): Update QA DB name for sept 1 release (#131) (ce4fa3f) * (DOCSP-32868): only boost 2 results + order results by score (#130) (5230992) * (DOCSP-32856) Add a toggle for streamed vs awaited responses to the demo (#126) (94e8a20) * (DOCSP-32808): k8s production readiness (#125) (2cdf539) * Upgrade azure openai (#128) (6b85e63) * (DOCSP-32759) [UI] Use text instead of a banner for "validate answers" (#124) (302b3c7) * (DOCSP-32604) More helpful max messages error message (#123) (1be4087) * (DOCSP-32254): Include information about latest product versions + minor prompt engineering (#116) (2e40123) * (DOCSP-32657): Remove same IP address validation on conversations (#112) (731ec20) * (DOCSP-31108): Add prod index scripts (#120) (3efe895) * (DOCSP-32807): Add pm2 for running server (#122) (f659e4b) * [Snyk] Security upgrade mongodb from 5.7.0 to 5.8.0 (#121) (c985fef) * Autoscaling fix (#119) (641199a) * Source GIT_COMMIT env variable (#95) (647b00a) * Allow DOP origins in staging (#117) (38a9b66) * (DOCSP-32074): Handle dev center page metadata (#96) (e283121) * Release chat-ui-v0.0.4 (d3bad60) * (DOCSP-32371): Allow for embedded content re-ingest after changes to chunking algorithm (#114) (8b9d68e) * (DOCSP-32279): Update prompts to not mention context when don't know answer (#113) (71ca44b) * (DOCSP-32688) Stop dereferencing spec (#115) (add0dd8) * (DOCSP-31438): Horizontal pod autoscaling (#110) (3c124d4) * fix registry (d52ff7d) * Release chat-ui-v0.0.3 (f8113b7) * Add chat-ui release-it config (731e3d6) * Switch Chatbot to default export (b5db7f0) * Chat UI v0.0.2 (#111) (7e5d684) * (DOCSP-32659): Make server call its own base url if not given a full URL path (#109) (6cdd026) * Allow users to override server URL via props (#108) (708e614) * (DOCSP-32547): Update prod env to use dedicated Azure deployment (#104) (ac93735) * Prod drone fixes (#107) (4df6432) * Update QA server DB (#105) (a12e7c9) * (DOCSP-32452): Dev Center data source remove `<img>` and `<div>` tags + youtube directive (#93) (c5dac2f) * (DOCSP-32596): Set up CORS on the server (#103) (76c9530) * (DOCSP-32104): Do not serve demo site in prod environment (#101) (5f0ea11) * (DOCSP-32247): Fix preprocessed content not added to DB (#100) (57cc24e) * fix handling of openapi specs (#99) (6f9b0e4) * fix hanging promise in async recursive operation (#98) (c974058) * Drone fix (#97) (e54a272) * (DOCSP-32363) [UI] handle cases where the LLM stops mid code example (#85) (9e26c7f) * (DOCSP-32155) [UI] UX Feedback (#81) (c34b246) * (DOCSP-32243) Handle and test pages with page-level code block select (#94) (f6c3f7e) * (DOCSP-32490) [UI] Link to current commit on GitHub (#90) (64dbd0d) * (DOCSP-32242): Handle tabbed Snooty content (#92) (f87f67d) * Restore 'tags' and use 'tagsIn' only internally (#91) (035f018) * (DOCSP-32227): Improve OpenAPI spec ingestion (#88) (f369218) * (DOCSP-31343): Add system diagram and info for ingest (#86) (6e4402b) * (DOCSP-32362): Improve conversation request logging (#89) (b251e0d) * (DOCSP-32253): Add more semantically relevant product names to metadata (#83) (5869f22) * (DOCSP-32434) [UI] Add error text for non-modal input (#87) (5de9282) * (DOCSP-32206): Strip comments from rst -> md (#84) (611a50d) * (DOCSP-32194): Fix drone and k8s for ingest/chat-server staging (#80) (b61c9bc) * Define engines to remove EBADENGINE warning (#82) (f333ef5) * (DOCSP-31440, DOCSP-32075): Add tags to chunk metadata (#78) (254ce2b) * (DOCSP-32217): Change chunker to chunk based on number of tokens + add max context tokens per message (#79) (e7c1cb6) * (DOCSP-32183 & DOCSP-32226): Automate test creation based on YAML file (#77) (30eb660) * Add database index creation scripts (#75) (53f296f) * (DOCSP-31575): LLM qualitative testing framework (#74) (f222e54) * (DOCSP-31111) [UI] publish react component to npm (#73) (3baf40e) * Refactor `ConversationService` class as `makeConversationService` func (#76) (6cb9a83) * (DOCSP-32070): LLM preprocessing on user queries (#72) (6ff4219) * (DOCSP-31925): Support tables in parsed snooty MD (#67) (539d403) * Reduce ChatGPT API flakiness + faster timeout when flaky (#69) (8938599) * Update Chat UI Packaging (#71) (2e5a0fb) * (DOCSP-32066): Q&A server reduce amount of backoff on embedding (#68) (fe938b0) * (DOCSP-31639): fix timeout err in tests (#70) (741ea29) * Implement page titles and prepending frontmatter to chunks (#64) (cf56a26) * Fix JS error surfaced in the UI (#66) (5e6866a) * bundle LG Chat with component lib (c4fc213) * Modify ui config for pre-release (135d211) * (DOCSP-31154) [UI] Handle timeout & "conversation too long" errors (#65) (1943608) * (DOCSP-31791): Boost manual in the results for short queries + make app config file (#63) (6449993) * (DOCSP-31523) [UI] Replace custom components with LG Chat (#61) (3ee640b) * Fix ingest drone + err handling (#59) (8307750) * Fix code block lang (#62) (cffcbf0) * Get current branch from Snooty Data API plus allow override (#56) (6114e7c) * Clean up pages for embedding content (#60) (45ce277) * (DOCSP-31799): Parse definition lists (and other relevant content inside of `"target"` nodes) (#58) (2d7d1b3) * rename chat-ui package (#57) (dc7794f) * (DOCSP-31574): Create qualitative testing env (#55) (0a1c92a) * Add tests for data deletion (#48) (9b1417a) * Update Snooty Data Source to strip 'index' (#50) (d6b0b2f) * Add ability to override Snooty project baseUrl (#52) (fccc8ca) * Split up tests + add utils (#53) (0dedc6f) * (DOCSP-31436) [Q&A] Investigate & improve streaming rate (#54) (cc90cd0) * (DOCSP-31120) [Q&A] separate further reading reference links from LLM response (#43) (a604d83) * (DOCSP-30863) [Q&A] type check requests (#23) (5f908c0) * Streamline/nitpick/bikeshed the PR template (#51) (28b4e1b) * (DOCSP-31601): Update server base URL (#49) (1c60a0b) * (DOCSP-31573): Update prompts to address bugs (#47) (a366803) * (DOCSP-30602): Ingest service prod pipeline (#42) (8e60755) * Update pull_request_template.md (ab166a7) * Monorepo docker build fix (#46) (1dcef28) * (DOCSP-31284): Integrate prod Snooty Data API (#44) (f80ce51) * (DOCSP-31360) [UI] Conditional Streaming (#45) (34490bb) * update root package-lock (578b06b) * rename chat-ui package (d7b6dcf) * Add shared dev script to monorepo root (67811d1) * Streaming works (#25) (c173ba2) * (DOCSP-30601): Create production release system (#41) (7472c9d) * (DOCSP-31304): Implement 'all' command (#40) (f0c4cf2) * (DOCSP-31201) [UI] Updated Legalese (#39) (39c0f97) * (DOCSP-30584): ingest service CI testing and staging drone pipeline (#38) (d2e8325) * Fix + improve devcenter source tests (#37) (be0a5eb) * Fix URL building for devcenter paths (#36) (4c75175) * Add logic for ingest resumption (#35) (0f650c2) * stricter LLM to reduce halucination risk + change demo page title (#33) (cfb4628) * (DOCSP-30890) [UI] Make component mobile responsive (#29) (66080a7) * Add minor optimization for rate-limited embed requests (#34) (7547e31) * (DOCSP-31121): Add request tracking across logs (#32) (7fdc1e5) * (DOCSP-30988): Implement ingest E2E (#28) (8467989) * (DOCSP-30843): IP Address tracking and validation (#21) (7335672) * Update seed-data script (4f7a2f7) * (DOCSP-31050): Fix CI tests + run tests on PR (#30) (74fe090) * (DOCSP-30953 & DOCSP-30945): Error and edge case handling (#24) (af1b845) * Add PR template to repo (#26) (5231ef0) * Fix tests (#27) (58599ea) * more specific linter targets (18d2f5d) * Move and refactor EmbeddedContent/Store and clean up core services (#19) (43078a2) * separate format from lint (3a817b7) * remove console logs (#22) (20027ce) * Add lists to LGMarkdown (#20) (14cf986) * (DOCSP-30964): Staging build (#18) (1163ff6) * Fix renames (945337e) * Rename "chunks" to "embedded content" (#17) (a18bb8d) * Implement Data Sources (#12) (e9c5784) * Move reusable pieces to chat-core (#14) (3babf35) * (DOCSP-30577): POST `/conversations/:conversationId/messages/:messageId/rating` endpoint (#16) (c659173) * E2E (#15) (c85a2aa) * failed message handling (16acff7) * better prompt validation error handling (6408669) * UI tweaks (f803938) * (DOCSP-30921) [UI] Add prompt validation & error handling (1) (cb98ec7) * (DOCSP-30616): `POST /conversations/:conversationId/messages/` route, no streaming (#13) (8451c08) * Add publishing pipeline (42ab17e) * (DOCSP-30575): POST /conversations/ route (#11) (d609171) * (DOCSP-30619, DOCSP-30758, DOCSP-30722): Add MongoDB integration and conversation/content services (#9) (020a501) * Implement ingest store/database (#8) (5b9f8f6) * Add initial design docs (#1) (31d9069) * (DOCSP-30585): Generate search index script (#10) (458403f) * Merge branch 'demodalize' of github.com:nlarew/docs-chatbot (8d46f37) * better avatars (4c7c712) * tweak card heading (40d1dff) * loading ui (58256da) * (DOCSP-30617): OpenAI LLM service (#7) (d4af1f9) * refactor + loading states (4418f8d) * (DOCSP-30618): Create embeddings service (#6) (c54d123) * mock out Q&A API (9d80c20) * Refactor to use a single chat input (6ac351a) * Animation works except for Card ref (2bc2b41) * Rename & refactor (b5385af) * Send button for all inputs with text (9ca1873) * Move input state up (664b456) * Animation works (b8c2f42) * Use card modal WIP (27c2392) * cleanup (d96bcf0) * Fix bad imports (524f896) * (DOCSP-30576): Stub out Q&A server (#4) (899c0a3) * LGMarkdown (33596cf) * Working on LGMarkdown (897b141) * consistent models with backend (319b2ae) * UI lint pass (d8051d9) * Leafie avatar (ebd660b) * Don't require React Context for conversations (86c90b6) * fix modal style (6ae1500) * Add storybook (73a4ec3) * Modal (cac03d2) * fix banner width (088d858) * Merge pull request #3 from mongodb/DOCSP-30565 (5264da5) * Update .drone.yml (a9e3ba5) * working staging build + updated untested prod (fe16f6c) * remove uncreated env secrets (318c501) * fix broken registry path (a1e1dff) * change release (072100f) * update environment info (ff22f4f) * empty state (a66ce15) * fix value file location (ee4f06b) * fix pipeline (89f9e8b) * Trigger build (408d7dc) * Add (broken) tests (329d9e0) * Trigger build (6cae68f) * Fix context (7d213c0) * update path to docker file (f0edac3) * trigger build (e5e64c0) * more debugging (2dfb919) * Use dependency injection for PageStore/ChunkStore interfaces (89577b8) * Remove start comment (287182d) * move drone file (7453f99) * trigger build? (95579f6) * test deploy pipeline (aab4632) * add cta (db01cdb) * apply prettier (c49d368) * remove unused imports (85f6ab3) * add prettier (c1cf166) * combined props (ae9d66c) * chat state works (e29ee25) * add temp aria-labelledby prop (a0d4695) * Scaffold the ingest CLI (81a5c41) * Update ingest/README (65d2902) * chatbot-ui -> chat-ui (4340d1a) * Merge pull request #2 from mongodben/DOCSP-30486 (17662e6) * Bootstrap ingest CLI tool (0590da2) * Add chatbot-ui (997bec5) * set up QA server project structure (7cd25d5) * Delete specs directory (6b06d68) * ready repo for first push (2b250da)
mongodben
added a commit
that referenced
this pull request
Nov 10, 2023
* (DOCSP-32482): Build out qualitative tests (#201) * touch up qualitative test suite + add tags * add phase 1 general draft * add general tests * remove console.log * add rename files * sept 23 question yaml generator * update scripts to handle different testing scenarios * add round 2 test draft * update test types * update more test cases * add all qualitative tests * fix bad case * (DOCSP-33744): update QA server DB name for latest snapshot (#210) update QA server DB name * dev QoL scripts * (DOCSP-32603): Add Terraform data source (#205) * add terraform data source * update for more pageS * implement CB feedback * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) * update repo to use lerna versioning * update contributing guide * update release conf * add release-it to chat-core * update version * Release mongodb-rag-core v0.0.1 * Release mongodb-rag-core v0.0.2-2 * add changelog setup * release notes/changelog * Add changelog * Release mongodb-rag-core v0.0.2-3 * Add changelog (4a9af69) * release notes/changelog (c9b3b0f) * add changelog setup (3e90095) * all * inside * add outside commit (shouldn't see) * inside (should see) * update release conf * remove from / to from github * Release mongodb-rag-core v0.0.2-4 * remove from / to from github (881e474) * update release conf (0477bc2) * inside (should see) (7f19587) * inside (c22271b) * all (90b568a) * configure so that publishes package from creation of gh tag via drone * add .drone pipelines to deploy new npm packages * Release mongodb-rag-core v0.0.2-5 * configure so that publishes package from creation of gh tag via drone (f9df609) * update ingest conf * update deps * add non prerelease version * Release mongodb-rag-ingest v0.1.3 * update core + ingest release conf * separate chat server package and implementation + update drone file * remove release-it from monorepo conf * update drone names * add new npm creds * update pub flow to speed up testing * Release mongodb-rag-core v0.0.2-6 * update core + ingest release conf (5559403) * update package files to use * update core version in server * working chat server * remove chat-server tests from prod build * remove chatbot-server-mongodb-public tests from prod build * make ingest tests pass * fix broken server tests * fix flaky tests * fix build issues * Release mongodb-rag-core v0.0.2-7 * add publish env vars * Release mongodb-rag-core v0.0.2-8 * trigger ingest staging build * hack drone to trigger builds * trigger ingest build again * retry drone * fix build commands * simplify dockerfile * make ingest a node executable * Release mongodb-rag-core v0.0.2 * update package.json files with files to include in release * Release mongodb-rag-core v0.0.3 * update package.json files with files to include in release (e4567ed) * Add simple readme to core * Release mongodb-rag-core v0.0.4 * Add simple readme to core (729f44a) * update ingest version of core to latest working * update monorepo chatcore version * Release mongodb-rag-ingest v0.1.4 * update ingest version of core to latest working (5a7b346) * update package.json files with files to include in release (e4567ed) * make ingest a node executable (da4011f) * fix build commands (37d746b) * retry drone (bbbdd82) * trigger ingest build again (1315732) * trigger ingest staging build (33c7f34) * fix flaky tests (9dc7153) * make ingest tests pass (00c9065) * update core + ingest release conf (5559403) * update dependencies to latest in mongodb ingest * update package-lock.json w new deps * update package-lock * trigger build of ingest-mongodb-public to test drone * make CLI entry point executable * update lock * Release mongodb-rag-ingest v0.1.5 * make CLI entry point executable (00b57d3) * update mongodb ingest to use latest ingest CLI * update executable path * update chat-server.dockerfile * Trigger staging build of mongodb server * update chat-core dependency * fix build err from test * update chatbot ui out destination * update npm chat-core publish config * Release mongodb-rag-core v0.0.5-0 * Release mongodb-rag-core v0.0.5-1 * Release mongodb-rag-core v0.0.5-0 (580e8ff) * fix drone script * Release mongodb-rag-core v0.0.5-2 * update drone publication pipelines * Release mongodb-rag-ingest v0.1.6-0 * fix drone file * Release mongodb-rag-ingest v0.1.6-1 * bunlde chat-core in ingest * update lock file * Release mongodb-rag-ingest v0.1.6-2 * bunlde chat-core in ingest (3f763fe) * Add missing typechat dep * make ingest reexport core * remove unused bundled dep * Release mongodb-rag-core v0.0.5-3 * Add missing typechat dep (4bf2594) * Release mongodb-rag-core v0.0.5 * update dependent packages * update lock * update lock * Release mongodb-rag-ingest v0.1.6 * update dependent packages (50a43db) * remove unused bundled dep (87f6f7e) * make ingest reexport core (5b0fd8e) * update drone * remove test from release-it b/c covered in CI * remove unused dep * update release conf * update core version * update lock * Release mongodb-chatbot-server v0.1.9 * update lock (e45350d) * update core version (7d15d35) * update release conf (05a6686) * remove unused dep (12c73a3) * remove test from release-it b/c covered in CI (99f2a9b) * update drone (932c9a8) * Release mongodb-rag-ingest v0.1.6 (d9e3ea9) * update lock (bd718e6) * update lock (dac1773) * update dependent packages (50a43db) * Release mongodb-rag-core v0.0.5 (55195cb) * Release mongodb-rag-core v0.0.5-3 (b765423) * remove unused bundled dep (87f6f7e) * make ingest reexport core (5b0fd8e) * Add missing typechat dep (4bf2594) * Release mongodb-rag-ingest v0.1.6-2 (a22fbdf) * update lock file (def3cb7) * bunlde chat-core in ingest (3f763fe) * Release mongodb-rag-ingest v0.1.6-1 (fdd536c) * fix drone file (9b16f20) * Release mongodb-rag-ingest v0.1.6-0 (767efac) * update drone publication pipelines (3d761a6) * Release mongodb-rag-core v0.0.5-2 (a5689a2) * fix drone script (f711595) * Release mongodb-rag-core v0.0.5-1 (0a3745a) * Release mongodb-rag-core v0.0.5-0 (580e8ff) * update npm chat-core publish config (5be4922) * update chatbot ui out destination (d3e20a2) * fix build err from test (af0cce0) * update chat-core dependency (069fb9b) * Trigger staging build of mongodb server (a438a82) * update chat-server.dockerfile (81999da) * update executable path (5b06c12) * update mongodb ingest to use latest ingest CLI (b21441f) * Release mongodb-rag-ingest v0.1.5 (bf852d1) * update lock (70a8c65) * make CLI entry point executable (00b57d3) * trigger build of ingest-mongodb-public to test drone (96b4206) * update package-lock (ff83848) * update package-lock.json w new deps (f4d6579) * update dependencies to latest in mongodb ingest (0d6e48a) * Release mongodb-rag-ingest v0.1.4 (7b7e8de) * update monorepo chatcore version (80ebd61) * update ingest version of core to latest working (5a7b346) * Release mongodb-rag-core v0.0.4 (e04b754) * Add simple readme to core (729f44a) * Release mongodb-rag-core v0.0.3 (230ab97) * update package.json files with files to include in release (e4567ed) * Release mongodb-rag-core v0.0.2 (e5e6ac1) * make ingest a node executable (da4011f) * simplify dockerfile (e2d8751) * fix build commands (37d746b) * retry drone (bbbdd82) * trigger ingest build again (1315732) * hack drone to trigger builds (88e3255) * trigger ingest staging build (33c7f34) * Release mongodb-rag-core v0.0.2-8 (2258731) * add publish env vars (1ed87ad) * Release mongodb-rag-core v0.0.2-7 (c5914d3) * fix build issues (965a9e0) * fix flaky tests (9dc7153) * fix broken server tests (e4b04b9) * make ingest tests pass (00c9065) * remove chatbot-server-mongodb-public tests from prod build (d95c984) * remove chat-server tests from prod build (4c0cd22) * working chat server (0248a11) * update core version in server (d236f91) * update package files to use (7d3e5b8) * Release mongodb-rag-core v0.0.2-6 (40688a7) * update pub flow to speed up testing (8e0409c) * add new npm creds (cf6084a) * update drone names (47dd13c) * remove release-it from monorepo conf (ff4434c) * separate chat server package and implementation + update drone file (4441cbf) * update core + ingest release conf (5559403) * Release mongodb-rag-ingest v0.1.3 (9a7a2f9) * add non prerelease version (756d1a7) * update deps (7f4ac98) * update ingest conf (88c47ee) * Release mongodb-rag-core v0.0.2-5 (ba238e6) * add .drone pipelines to deploy new npm packages (f4a8210) * configure so that publishes package from creation of gh tag via drone (f9df609) * Release mongodb-rag-core v0.0.2-4 (d626d91) * remove from / to from github (881e474) * update release conf (0477bc2) * inside (should see) (7f19587) * add outside commit (shouldn't see) (feccfe7) * inside (c22271b) * all (90b568a) * Release mongodb-rag-core v0.0.2-3 (a90c1ba) * Add changelog (4a9af69) * release notes/changelog (c9b3b0f) * add changelog setup (3e90095) * Release mongodb-rag-core v0.0.2-2 (bcf3c9c) * Release mongodb-rag-core v0.0.1 (a3a1355) * update version (f6e1d8f) * add release-it to chat-core (629de3e) * update release conf (73308a6) * update contributing guide (11ee87b) * update repo to use lerna versioning (f7c7285) * Merge remote-tracking branch 'upstream/main' into oss-refactor-pt-2 (412da6a) * Ingest plugins (#211) (0206bee) * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) (b8541a7) * OSS Chat Server (#206) (820e249) * (DOCSP-32603): Add Terraform data source (#205) (0d64743) * dev QoL scripts (3813ad7) * (DOCSP-33744): update QA server DB name for latest snapshot (#210) (5df2bf0) * (DOCSP-32482): Build out qualitative tests (#201) (0535a89) * Organize some files (#207) (f8cdd3c) * Conversations refactor (#199) (5a60a15) * Release ingest-v0.1.2 (#204) (ed34507) * [Snyk] Upgrade urllib from 3.19.1 to 3.19.2 (#203) (743cc01) * Add Rust and Entity Framework Snooty data sources (#202) (ff15139) * [Snyk] Upgrade urllib from 3.19.0 to 3.19.1 (#200) (dea05e0) * Release chat-server v0.1.3 (ac542c8) * Use GITHUB_TOKEN for chat-ui releases (38a9e1a) * [Snyk] Upgrade urllib from 3.18.1 to 3.19.0 (#197) (dd2baaf) * Do not fail query on rephrase fail (#196) (8b7c60a) * Release chat-ui v0.0.8 (03bf7cb) * Bundle new @lg-chat components (2f1f11b) * Release chat-ui v0.0.7 (d082a63) * Unify release-it config (419422e) * Release chat-server v0.1.2 (b0b425c) * [Snyk] Upgrade urllib from 3.18.0 to 3.18.1 (#194) (2c98ca4) * [DOCSP-32950] Create a DevCenter Chatbot Component (#193) (c219d03) * (DOCSP-33445) [ANALYTICS] Scrubbed Views (#195) (8ef2bd7) * (DOCSP-33419) [ANALYTICS] Operational Views (#192) (aae8259) * Release scripts v0.1.2 (fbf2bbf) * Add whether message was deemed "Do not answer" to conversation (#191) (955b722) * (DOCSP-31104) [UI] Add tests to FE services (#190) (d3e2000) * Release ingest v0.1.1 (5b06cd2) * Update package-lock.json (a568d00) * Replace GitHub data source (#188) (747d08e) * Update env vars for analyze script (0a46085) * Add scrubbed message analyzer (#187) (9cef6b8) * (DOCSP-33291) [UI] Add tck tracking to links (#185) (acfb6a1) * Update README.md (a38cbf7) * Release 0.1.1 (8ee27c1) * Fix typo in scripts build (0534458) * Release 0.1.0 (09fa262) * Add release-it to scripts (9ea2e71) * Trigger drone run for scripts service (b1ffcc8) * Fix drone scripts (#186) (9bae8e9) * Cron scrub (#184) (d67dda9) * (DOCSP-33273) Open Source Licensing (#183) (00611bf) * De-flake SnootyProjectsInfo test (#181) (9f7250e) * (DOCSP-32930): Add pymongo metadata (#179) (6694209) * Update release-it git add config to add monorepo stuff (679e447) * Update package-lock in monorepo for release (e7308ef) * Release 0.1.1 (d60d14b) * Update release-it pre-hooks (718b96d) * Run npm audit fix (#178) (c74e1a9) * Remove `seed-content` package (#157) (1b9ac43) * Add scrub message script (#177) (ee2f864) * Remove extra padding around initial input (9dc27ad) * (DOCSP-33094): Save conversation embeddings (#175) (9c6e79d) * Ingest release 0.1.0 (#174) (4928eed) * Release chat-ui-v0.0.6 (2bf2a0e) * (DOCSP-33059) [UI] Fix suggested prompts & other LG Chat tweaks (#171) (cde1b74) * chat server v0.1.0 (#170) (ba69ad8) * new commit for build (56ed01e) * trigger ingest build (4052af9) * clean up spec + remove ip address validation from rate msg (#166) (28cf222) * UI Tweaks (#159) (53b392f) * Delete chat-server/trigger.me (#165) (5542f41) * update repo name in code (#163) (f9628bb) * Update README.md (5ad9cd0) * Launch pre releases (#162) (9b2380b) * Expand Contributing Guide and `ingest` README (#161) (30434e3) * filter out very small chunks (#160) (d952c6d) * (DOCSP-31928): Implement numbered lists (#154) (f41258d) * clarify ingest readme (115b00b) * move git install to correct docker stage (40600cb) * (DOCSP-32991, DOCSP-32992): `GitDataSource`, functionality for converting HTML to Markdown, ingest Scala, Java Reactive Streams, and C docs (#152) (7118444) * (DOCSP-32915) [UI] Streamer bugs out when switching tabs (#156) (91c288d) * (DOCSP-33090) [UI] Remove experimental badge from input on focus (#155) (a3ea236) * (DOCSP-32993, DOCSP-32906, DOCSP-32993, DOCSP-32990): Data sources for MD on Github + data ingestion of related content (#148) (7daed63) * change recommended prompt (#150) (2521f08) * add missing env example (#153) (1201a34) * [Snyk] Upgrade mongodb from 5.8.0 to 5.8.1 (#151) (966a4bf) * make preprocessor less biased toward shell (#149) (886cbe5) * (DOCSP-32616): Refactor config, add library, and log config (#143) (4fcfc58) * add default + remove unused style (0296b42) * Update chat-ui release config (3174b61) * Release chat-ui-v0.0.5 (65727f7) * Add root release:chat-ui command (4621989) * (DOCSP-32757) [UI] Update to Responsive LG Chat Components (#145) (81b70bb) * (DOCSP-32670): Basic server load testing (#136) (282c947) * [Snyk] Upgrade urllib from 3.17.2 to 3.18.0 (#147) (aefa9e0) * (DOCSP-32756): Implement headings in tables (#140) (ea416ce) * custom/synthetic/meta data source content (#127) (d687312) * [Snyk] Upgrade urllib from 3.17.1 to 3.17.2 (#144) (209135a) * (DOCSP-32673) [UI] Suppress focus to prevent modal reopen (#139) (43b8a98) * (DOCSP-32936): Increase single IP rate limits to be more line w firewall rate limits (#141) (87d8785) * (DOCSP-31863): Fix flaky test, refactor core llm service + server implementation (#142) (584a5e3) * (DOCSP-32916): Conversation service rate limiting + slow down (#138) (a88a5d5) * (DOCSP-32914): Stop unindenting context (#137) (27630c6) * (DOCSP-31285): Ingest PyMongo docs (#118) (e04c10b) * (DOCSP-32694) Dark mode: use LG palette & provider + cleanup (a95f5c5) * (DOCSP-32694) [UI] Add darkMode prop (#129) (820229c) * (DOCSP-32896) Remove mention of <CONTEXT> to reduce damage of prompt leakage (#135) (1484426) * Update internal docs (#134) (a3b2f56) * Handle Snooty Data API tombstones (#133) (a775da4) * (DOCSP-32867): Update QA DB name for sept 1 release (#131) (ce4fa3f) * (DOCSP-32868): only boost 2 results + order results by score (#130) (5230992) * (DOCSP-32856) Add a toggle for streamed vs awaited responses to the demo (#126) (94e8a20) * (DOCSP-32808): k8s production readiness (#125) (2cdf539) * Upgrade azure openai (#128) (6b85e63) * (DOCSP-32759) [UI] Use text instead of a banner for "validate answers" (#124) (302b3c7) * (DOCSP-32604) More helpful max messages error message (#123) (1be4087) * (DOCSP-32254): Include information about latest product versions + minor prompt engineering (#116) (2e40123) * (DOCSP-32657): Remove same IP address validation on conversations (#112) (731ec20) * (DOCSP-31108): Add prod index scripts (#120) (3efe895) * (DOCSP-32807): Add pm2 for running server (#122) (f659e4b) * [Snyk] Security upgrade mongodb from 5.7.0 to 5.8.0 (#121) (c985fef) * Autoscaling fix (#119) (641199a) * Source GIT_COMMIT env variable (#95) (647b00a) * Allow DOP origins in staging (#117) (38a9b66) * (DOCSP-32074): Handle dev center page metadata (#96) (e283121) * Release chat-ui-v0.0.4 (d3bad60) * (DOCSP-32371): Allow for embedded content re-ingest after changes to chunking algorithm (#114) (8b9d68e) * (DOCSP-32279): Update prompts to not mention context when don't know answer (#113) (71ca44b) * (DOCSP-32688) Stop dereferencing spec (#115) (add0dd8) * (DOCSP-31438): Horizontal pod autoscaling (#110) (3c124d4) * fix registry (d52ff7d) * Release chat-ui-v0.0.3 (f8113b7) * Add chat-ui release-it config (731e3d6) * Switch Chatbot to default export (b5db7f0) * Chat UI v0.0.2 (#111) (7e5d684) * (DOCSP-32659): Make server call its own base url if not given a full URL path (#109) (6cdd026) * Allow users to override server URL via props (#108) (708e614) * (DOCSP-32547): Update prod env to use dedicated Azure deployment (#104) (ac93735) * Prod drone fixes (#107) (4df6432) * Update QA server DB (#105) (a12e7c9) * (DOCSP-32452): Dev Center data source remove `<img>` and `<div>` tags + youtube directive (#93) (c5dac2f) * (DOCSP-32596): Set up CORS on the server (#103) (76c9530) * (DOCSP-32104): Do not serve demo site in prod environment (#101) (5f0ea11) * (DOCSP-32247): Fix preprocessed content not added to DB (#100) (57cc24e) * fix handling of openapi specs (#99) (6f9b0e4) * fix hanging promise in async recursive operation (#98) (c974058) * Drone fix (#97) (e54a272) * (DOCSP-32363) [UI] handle cases where the LLM stops mid code example (#85) (9e26c7f) * (DOCSP-32155) [UI] UX Feedback (#81) (c34b246) * (DOCSP-32243) Handle and test pages with page-level code block select (#94) (f6c3f7e) * (DOCSP-32490) [UI] Link to current commit on GitHub (#90) (64dbd0d) * (DOCSP-32242): Handle tabbed Snooty content (#92) (f87f67d) * Restore 'tags' and use 'tagsIn' only internally (#91) (035f018) * (DOCSP-32227): Improve OpenAPI spec ingestion (#88) (f369218) * (DOCSP-31343): Add system diagram and info for ingest (#86) (6e4402b) * (DOCSP-32362): Improve conversation request logging (#89) (b251e0d) * (DOCSP-32253): Add more semantically relevant product names to metadata (#83) (5869f22) * (DOCSP-32434) [UI] Add error text for non-modal input (#87) (5de9282) * (DOCSP-32206): Strip comments from rst -> md (#84) (611a50d) * (DOCSP-32194): Fix drone and k8s for ingest/chat-server staging (#80) (b61c9bc) * Define engines to remove EBADENGINE warning (#82) (f333ef5) * (DOCSP-31440, DOCSP-32075): Add tags to chunk metadata (#78) (254ce2b) * (DOCSP-32217): Change chunker to chunk based on number of tokens + add max context tokens per message (#79) (e7c1cb6) * (DOCSP-32183 & DOCSP-32226): Automate test creation based on YAML file (#77) (30eb660) * Add database index creation scripts (#75) (53f296f) * (DOCSP-31575): LLM qualitative testing framework (#74) (f222e54) * (DOCSP-31111) [UI] publish react component to npm (#73) (3baf40e) * Refactor `ConversationService` class as `makeConversationService` func (#76) (6cb9a83) * (DOCSP-32070): LLM preprocessing on user queries (#72) (6ff4219) * (DOCSP-31925): Support tables in parsed snooty MD (#67) (539d403) * Reduce ChatGPT API flakiness + faster timeout when flaky (#69) (8938599) * Update Chat UI Packaging (#71) (2e5a0fb) * (DOCSP-32066): Q&A server reduce amount of backoff on embedding (#68) (fe938b0) * (DOCSP-31639): fix timeout err in tests (#70) (741ea29) * Implement page titles and prepending frontmatter to chunks (#64) (cf56a26) * Fix JS error surfaced in the UI (#66) (5e6866a) * bundle LG Chat with component lib (c4fc213) * Modify ui config for pre-release (135d211) * (DOCSP-31154) [UI] Handle timeout & "conversation too long" errors (#65) (1943608) * (DOCSP-31791): Boost manual in the results for short queries + make app config file (#63) (6449993) * (DOCSP-31523) [UI] Replace custom components with LG Chat (#61) (3ee640b) * Fix ingest drone + err handling (#59) (8307750) * Fix code block lang (#62) (cffcbf0) * Get current branch from Snooty Data API plus allow override (#56) (6114e7c) * Clean up pages for embedding content (#60) (45ce277) * (DOCSP-31799): Parse definition lists (and other relevant content inside of `"target"` nodes) (#58) (2d7d1b3) * rename chat-ui package (#57) (dc7794f) * (DOCSP-31574): Create qualitative testing env (#55) (0a1c92a) * Add tests for data deletion (#48) (9b1417a) * Update Snooty Data Source to strip 'index' (#50) (d6b0b2f) * Add ability to override Snooty project baseUrl (#52) (fccc8ca) * Split up tests + add utils (#53) (0dedc6f) * (DOCSP-31436) [Q&A] Investigate & improve streaming rate (#54) (cc90cd0) * (DOCSP-31120) [Q&A] separate further reading reference links from LLM response (#43) (a604d83) * (DOCSP-30863) [Q&A] type check requests (#23) (5f908c0) * Streamline/nitpick/bikeshed the PR template (#51) (28b4e1b) * (DOCSP-31601): Update server base URL (#49) (1c60a0b) * (DOCSP-31573): Update prompts to address bugs (#47) (a366803) * (DOCSP-30602): Ingest service prod pipeline (#42) (8e60755) * Update pull_request_template.md (ab166a7) * Monorepo docker build fix (#46) (1dcef28) * (DOCSP-31284): Integrate prod Snooty Data API (#44) (f80ce51) * (DOCSP-31360) [UI] Conditional Streaming (#45) (34490bb) * update root package-lock (578b06b) * rename chat-ui package (d7b6dcf) * Add shared dev script to monorepo root (67811d1) * Streaming works (#25) (c173ba2) * (DOCSP-30601): Create production release system (#41) (7472c9d) * (DOCSP-31304): Implement 'all' command (#40) (f0c4cf2) * (DOCSP-31201) [UI] Updated Legalese (#39) (39c0f97) * (DOCSP-30584): ingest service CI testing and staging drone pipeline (#38) (d2e8325) * Fix + improve devcenter source tests (#37) (be0a5eb) * Fix URL building for devcenter paths (#36) (4c75175) * Add logic for ingest resumption (#35) (0f650c2) * stricter LLM to reduce halucination risk + change demo page title (#33) (cfb4628) * (DOCSP-30890) [UI] Make component mobile responsive (#29) (66080a7) * Add minor optimization for rate-limited embed requests (#34) (7547e31) * (DOCSP-31121): Add request tracking across logs (#32) (7fdc1e5) * (DOCSP-30988): Implement ingest E2E (#28) (8467989) * (DOCSP-30843): IP Address tracking and validation (#21) (7335672) * Update seed-data script (4f7a2f7) * (DOCSP-31050): Fix CI tests + run tests on PR (#30) (74fe090) * (DOCSP-30953 & DOCSP-30945): Error and edge case handling (#24) (af1b845) * Add PR template to repo (#26) (5231ef0) * Fix tests (#27) (58599ea) * more specific linter targets (18d2f5d) * Move and refactor EmbeddedContent/Store and clean up core services (#19) (43078a2) * separate format from lint (3a817b7) * remove console logs (#22) (20027ce) * Add lists to LGMarkdown (#20) (14cf986) * (DOCSP-30964): Staging build (#18) (1163ff6) * Fix renames (945337e) * Rename "chunks" to "embedded content" (#17) (a18bb8d) * Implement Data Sources (#12) (e9c5784) * Move reusable pieces to chat-core (#14) (3babf35) * (DOCSP-30577): POST `/conversations/:conversationId/messages/:messageId/rating` endpoint (#16) (c659173) * E2E (#15) (c85a2aa) * failed message handling (16acff7) * better prompt validation error handling (6408669) * UI tweaks (f803938) * (DOCSP-30921) [UI] Add prompt validation & error handling (1) (cb98ec7) * (DOCSP-30616): `POST /conversations/:conversationId/messages/` route, no streaming (#13) (8451c08) * Add publishing pipeline (42ab17e) * (DOCSP-30575): POST /conversations/ route (#11) (d609171) * (DOCSP-30619, DOCSP-30758, DOCSP-30722): Add MongoDB integration and conversation/content services (#9) (020a501) * Implement ingest store/database (#8) (5b9f8f6) * Add initial design docs (#1) (31d9069) * (DOCSP-30585): Generate search index script (#10) (458403f) * Merge branch 'demodalize' of github.com:nlarew/docs-chatbot (8d46f37) * better avatars (4c7c712) * tweak card heading (40d1dff) * loading ui (58256da) * (DOCSP-30617): OpenAI LLM service (#7) (d4af1f9) * refactor + loading states (4418f8d) * (DOCSP-30618): Create embeddings service (#6) (c54d123) * mock out Q&A API (9d80c20) * Refactor to use a single chat input (6ac351a) * Animation works except for Card ref (2bc2b41) * Rename & refactor (b5385af) * Send button for all inputs with text (9ca1873) * Move input state up (664b456) * Animation works (b8c2f42) * Use card modal WIP (27c2392) * cleanup (d96bcf0) * Fix bad imports (524f896) * (DOCSP-30576): Stub out Q&A server (#4) (899c0a3) * LGMarkdown (33596cf) * Working on LGMarkdown (897b141) * consistent models with backend (319b2ae) * UI lint pass (d8051d9) * Leafie avatar (ebd660b) * Don't require React Context for conversations (86c90b6) * fix modal style (6ae1500) * Add storybook (73a4ec3) * Modal (cac03d2) * fix banner width (088d858) * Merge pull request #3 from mongodb/DOCSP-30565 (5264da5) * Update .drone.yml (a9e3ba5) * working staging build + updated untested prod (fe16f6c) * remove uncreated env secrets (318c501) * fix broken registry path (a1e1dff) * change release (072100f) * update environment info (ff22f4f) * empty state (a66ce15) * fix value file location (ee4f06b) * fix pipeline (89f9e8b) * Trigger build (408d7dc) * Add (broken) tests (329d9e0) * Trigger build (6cae68f) * Fix context (7d213c0) * update path to docker file (f0edac3) * trigger build (e5e64c0) * more debugging (2dfb919) * Use dependency injection for PageStore/ChunkStore interfaces (89577b8) * Remove start comment (287182d) * move drone file (7453f99) * trigger build? (95579f6) * test deploy pipeline (aab4632) * add cta (db01cdb) * apply prettier (c49d368) * remove unused imports (85f6ab3) * add prettier (c1cf166) * combined props (ae9d66c) * chat state works (e29ee25) * add temp aria-labelledby prop (a0d4695) * Scaffold the ingest CLI (81a5c41) * Update ingest/README (65d2902) * chatbot-ui -> chat-ui (4340d1a) * Merge pull request #2 from mongodben/DOCSP-30486 (17662e6) * Bootstrap ingest CLI tool (0590da2) * Add chatbot-ui (997bec5) * set up QA server project structure (7cd25d5) * Delete specs directory (6b06d68) * ready repo for first push (2b250da) * refactor chatbot server to use latest versions * remove trigger files * clean drone up after changes for development * update contributor guide * Update CONTRIBUTING.md Co-authored-by: Chris Bush <chris.bush@mongodb.com> * Update chat-core/src/extractFrontMatter.ts --------- Co-authored-by: Nick Larew <nick.larew@mongodb.com> Co-authored-by: Chris Bush <chris.bush@mongodb.com>
mongodben
added a commit
that referenced
this pull request
Nov 14, 2023
* Organize some files (#207) * Organize some files * Move test_data out of source * Undo over-barreling * Create test helper directory * OSS Chat Server (#206) * add OSS_TODOs * Add comments * more comment edits * fix build errs * remove abstraction from chat-core * update mongodb imports * Fix broken tests * start OpenAiEmbedFunc refactor * split data stores * update ingest w new stores * configurable reference links * Make max number messages & tokens in single RAG in conversation configurable * update tests w new route conf * update SomeStreamEventUsage * Refactor project as lib * wrap close statements so second doesn't fail even if first does * implement CB feedback * add gitignore * Ingest plugins (#211) * Ingest plugins WIP Externalize config WIP EmbedFunc -> Embedder Continue dev Fix test Update dockerfile and cron jobs Documentation Partialize configs * Fix test * WIP * Cleanup * Add testing * Further work to make it work? * Tidy * Update ingest/src/Config.ts Co-authored-by: Ben Perlmutter <90647379+mongodben@users.noreply.github.com> * Minor fix * Fix build? * Update Config usage * Address comments --------- Co-authored-by: Ben Perlmutter <90647379+mongodben@users.noreply.github.com> * OSS Refactor: Set up monorepo for package publication (#232) * (DOCSP-32482): Build out qualitative tests (#201) * touch up qualitative test suite + add tags * add phase 1 general draft * add general tests * remove console.log * add rename files * sept 23 question yaml generator * update scripts to handle different testing scenarios * add round 2 test draft * update test types * update more test cases * add all qualitative tests * fix bad case * (DOCSP-33744): update QA server DB name for latest snapshot (#210) update QA server DB name * dev QoL scripts * (DOCSP-32603): Add Terraform data source (#205) * add terraform data source * update for more pageS * implement CB feedback * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) * update repo to use lerna versioning * update contributing guide * update release conf * add release-it to chat-core * update version * Release mongodb-rag-core v0.0.1 * Release mongodb-rag-core v0.0.2-2 * add changelog setup * release notes/changelog * Add changelog * Release mongodb-rag-core v0.0.2-3 * Add changelog (4a9af69) * release notes/changelog (c9b3b0f) * add changelog setup (3e90095) * all * inside * add outside commit (shouldn't see) * inside (should see) * update release conf * remove from / to from github * Release mongodb-rag-core v0.0.2-4 * remove from / to from github (881e474) * update release conf (0477bc2) * inside (should see) (7f19587) * inside (c22271b) * all (90b568a) * configure so that publishes package from creation of gh tag via drone * add .drone pipelines to deploy new npm packages * Release mongodb-rag-core v0.0.2-5 * configure so that publishes package from creation of gh tag via drone (f9df609) * update ingest conf * update deps * add non prerelease version * Release mongodb-rag-ingest v0.1.3 * update core + ingest release conf * separate chat server package and implementation + update drone file * remove release-it from monorepo conf * update drone names * add new npm creds * update pub flow to speed up testing * Release mongodb-rag-core v0.0.2-6 * update core + ingest release conf (5559403) * update package files to use * update core version in server * working chat server * remove chat-server tests from prod build * remove chatbot-server-mongodb-public tests from prod build * make ingest tests pass * fix broken server tests * fix flaky tests * fix build issues * Release mongodb-rag-core v0.0.2-7 * add publish env vars * Release mongodb-rag-core v0.0.2-8 * trigger ingest staging build * hack drone to trigger builds * trigger ingest build again * retry drone * fix build commands * simplify dockerfile * make ingest a node executable * Release mongodb-rag-core v0.0.2 * update package.json files with files to include in release * Release mongodb-rag-core v0.0.3 * update package.json files with files to include in release (e4567ed) * Add simple readme to core * Release mongodb-rag-core v0.0.4 * Add simple readme to core (729f44a) * update ingest version of core to latest working * update monorepo chatcore version * Release mongodb-rag-ingest v0.1.4 * update ingest version of core to latest working (5a7b346) * update package.json files with files to include in release (e4567ed) * make ingest a node executable (da4011f) * fix build commands (37d746b) * retry drone (bbbdd82) * trigger ingest build again (1315732) * trigger ingest staging build (33c7f34) * fix flaky tests (9dc7153) * make ingest tests pass (00c9065) * update core + ingest release conf (5559403) * update dependencies to latest in mongodb ingest * update package-lock.json w new deps * update package-lock * trigger build of ingest-mongodb-public to test drone * make CLI entry point executable * update lock * Release mongodb-rag-ingest v0.1.5 * make CLI entry point executable (00b57d3) * update mongodb ingest to use latest ingest CLI * update executable path * update chat-server.dockerfile * Trigger staging build of mongodb server * update chat-core dependency * fix build err from test * update chatbot ui out destination * update npm chat-core publish config * Release mongodb-rag-core v0.0.5-0 * Release mongodb-rag-core v0.0.5-1 * Release mongodb-rag-core v0.0.5-0 (580e8ff) * fix drone script * Release mongodb-rag-core v0.0.5-2 * update drone publication pipelines * Release mongodb-rag-ingest v0.1.6-0 * fix drone file * Release mongodb-rag-ingest v0.1.6-1 * bunlde chat-core in ingest * update lock file * Release mongodb-rag-ingest v0.1.6-2 * bunlde chat-core in ingest (3f763fe) * Add missing typechat dep * make ingest reexport core * remove unused bundled dep * Release mongodb-rag-core v0.0.5-3 * Add missing typechat dep (4bf2594) * Release mongodb-rag-core v0.0.5 * update dependent packages * update lock * update lock * Release mongodb-rag-ingest v0.1.6 * update dependent packages (50a43db) * remove unused bundled dep (87f6f7e) * make ingest reexport core (5b0fd8e) * update drone * remove test from release-it b/c covered in CI * remove unused dep * update release conf * update core version * update lock * Release mongodb-chatbot-server v0.1.9 * update lock (e45350d) * update core version (7d15d35) * update release conf (05a6686) * remove unused dep (12c73a3) * remove test from release-it b/c covered in CI (99f2a9b) * update drone (932c9a8) * Release mongodb-rag-ingest v0.1.6 (d9e3ea9) * update lock (bd718e6) * update lock (dac1773) * update dependent packages (50a43db) * Release mongodb-rag-core v0.0.5 (55195cb) * Release mongodb-rag-core v0.0.5-3 (b765423) * remove unused bundled dep (87f6f7e) * make ingest reexport core (5b0fd8e) * Add missing typechat dep (4bf2594) * Release mongodb-rag-ingest v0.1.6-2 (a22fbdf) * update lock file (def3cb7) * bunlde chat-core in ingest (3f763fe) * Release mongodb-rag-ingest v0.1.6-1 (fdd536c) * fix drone file (9b16f20) * Release mongodb-rag-ingest v0.1.6-0 (767efac) * update drone publication pipelines (3d761a6) * Release mongodb-rag-core v0.0.5-2 (a5689a2) * fix drone script (f711595) * Release mongodb-rag-core v0.0.5-1 (0a3745a) * Release mongodb-rag-core v0.0.5-0 (580e8ff) * update npm chat-core publish config (5be4922) * update chatbot ui out destination (d3e20a2) * fix build err from test (af0cce0) * update chat-core dependency (069fb9b) * Trigger staging build of mongodb server (a438a82) * update chat-server.dockerfile (81999da) * update executable path (5b06c12) * update mongodb ingest to use latest ingest CLI (b21441f) * Release mongodb-rag-ingest v0.1.5 (bf852d1) * update lock (70a8c65) * make CLI entry point executable (00b57d3) * trigger build of ingest-mongodb-public to test drone (96b4206) * update package-lock (ff83848) * update package-lock.json w new deps (f4d6579) * update dependencies to latest in mongodb ingest (0d6e48a) * Release mongodb-rag-ingest v0.1.4 (7b7e8de) * update monorepo chatcore version (80ebd61) * update ingest version of core to latest working (5a7b346) * Release mongodb-rag-core v0.0.4 (e04b754) * Add simple readme to core (729f44a) * Release mongodb-rag-core v0.0.3 (230ab97) * update package.json files with files to include in release (e4567ed) * Release mongodb-rag-core v0.0.2 (e5e6ac1) * make ingest a node executable (da4011f) * simplify dockerfile (e2d8751) * fix build commands (37d746b) * retry drone (bbbdd82) * trigger ingest build again (1315732) * hack drone to trigger builds (88e3255) * trigger ingest staging build (33c7f34) * Release mongodb-rag-core v0.0.2-8 (2258731) * add publish env vars (1ed87ad) * Release mongodb-rag-core v0.0.2-7 (c5914d3) * fix build issues (965a9e0) * fix flaky tests (9dc7153) * fix broken server tests (e4b04b9) * make ingest tests pass (00c9065) * remove chatbot-server-mongodb-public tests from prod build (d95c984) * remove chat-server tests from prod build (4c0cd22) * working chat server (0248a11) * update core version in server (d236f91) * update package files to use (7d3e5b8) * Release mongodb-rag-core v0.0.2-6 (40688a7) * update pub flow to speed up testing (8e0409c) * add new npm creds (cf6084a) * update drone names (47dd13c) * remove release-it from monorepo conf (ff4434c) * separate chat server package and implementation + update drone file (4441cbf) * update core + ingest release conf (5559403) * Release mongodb-rag-ingest v0.1.3 (9a7a2f9) * add non prerelease version (756d1a7) * update deps (7f4ac98) * update ingest conf (88c47ee) * Release mongodb-rag-core v0.0.2-5 (ba238e6) * add .drone pipelines to deploy new npm packages (f4a8210) * configure so that publishes package from creation of gh tag via drone (f9df609) * Release mongodb-rag-core v0.0.2-4 (d626d91) * remove from / to from github (881e474) * update release conf (0477bc2) * inside (should see) (7f19587) * add outside commit (shouldn't see) (feccfe7) * inside (c22271b) * all (90b568a) * Release mongodb-rag-core v0.0.2-3 (a90c1ba) * Add changelog (4a9af69) * release notes/changelog (c9b3b0f) * add changelog setup (3e90095) * Release mongodb-rag-core v0.0.2-2 (bcf3c9c) * Release mongodb-rag-core v0.0.1 (a3a1355) * update version (f6e1d8f) * add release-it to chat-core (629de3e) * update release conf (73308a6) * update contributing guide (11ee87b) * update repo to use lerna versioning (f7c7285) * Merge remote-tracking branch 'upstream/main' into oss-refactor-pt-2 (412da6a) * Ingest plugins (#211) (0206bee) * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) (b8541a7) * OSS Chat Server (#206) (820e249) * (DOCSP-32603): Add Terraform data source (#205) (0d64743) * dev QoL scripts (3813ad7) * (DOCSP-33744): update QA server DB name for latest snapshot (#210) (5df2bf0) * (DOCSP-32482): Build out qualitative tests (#201) (0535a89) * Organize some files (#207) (f8cdd3c) * Conversations refactor (#199) (5a60a15) * Release ingest-v0.1.2 (#204) (ed34507) * [Snyk] Upgrade urllib from 3.19.1 to 3.19.2 (#203) (743cc01) * Add Rust and Entity Framework Snooty data sources (#202) (ff15139) * [Snyk] Upgrade urllib from 3.19.0 to 3.19.1 (#200) (dea05e0) * Release chat-server v0.1.3 (ac542c8) * Use GITHUB_TOKEN for chat-ui releases (38a9e1a) * [Snyk] Upgrade urllib from 3.18.1 to 3.19.0 (#197) (dd2baaf) * Do not fail query on rephrase fail (#196) (8b7c60a) * Release chat-ui v0.0.8 (03bf7cb) * Bundle new @lg-chat components (2f1f11b) * Release chat-ui v0.0.7 (d082a63) * Unify release-it config (419422e) * Release chat-server v0.1.2 (b0b425c) * [Snyk] Upgrade urllib from 3.18.0 to 3.18.1 (#194) (2c98ca4) * [DOCSP-32950] Create a DevCenter Chatbot Component (#193) (c219d03) * (DOCSP-33445) [ANALYTICS] Scrubbed Views (#195) (8ef2bd7) * (DOCSP-33419) [ANALYTICS] Operational Views (#192) (aae8259) * Release scripts v0.1.2 (fbf2bbf) * Add whether message was deemed "Do not answer" to conversation (#191) (955b722) * (DOCSP-31104) [UI] Add tests to FE services (#190) (d3e2000) * Release ingest v0.1.1 (5b06cd2) * Update package-lock.json (a568d00) * Replace GitHub data source (#188) (747d08e) * Update env vars for analyze script (0a46085) * Add scrubbed message analyzer (#187) (9cef6b8) * (DOCSP-33291) [UI] Add tck tracking to links (#185) (acfb6a1) * Update README.md (a38cbf7) * Release 0.1.1 (8ee27c1) * Fix typo in scripts build (0534458) * Release 0.1.0 (09fa262) * Add release-it to scripts (9ea2e71) * Trigger drone run for scripts service (b1ffcc8) * Fix drone scripts (#186) (9bae8e9) * Cron scrub (#184) (d67dda9) * (DOCSP-33273) Open Source Licensing (#183) (00611bf) * De-flake SnootyProjectsInfo test (#181) (9f7250e) * (DOCSP-32930): Add pymongo metadata (#179) (6694209) * Update release-it git add config to add monorepo stuff (679e447) * Update package-lock in monorepo for release (e7308ef) * Release 0.1.1 (d60d14b) * Update release-it pre-hooks (718b96d) * Run npm audit fix (#178) (c74e1a9) * Remove `seed-content` package (#157) (1b9ac43) * Add scrub message script (#177) (ee2f864) * Remove extra padding around initial input (9dc27ad) * (DOCSP-33094): Save conversation embeddings (#175) (9c6e79d) * Ingest release 0.1.0 (#174) (4928eed) * Release chat-ui-v0.0.6 (2bf2a0e) * (DOCSP-33059) [UI] Fix suggested prompts & other LG Chat tweaks (#171) (cde1b74) * chat server v0.1.0 (#170) (ba69ad8) * new commit for build (56ed01e) * trigger ingest build (4052af9) * clean up spec + remove ip address validation from rate msg (#166) (28cf222) * UI Tweaks (#159) (53b392f) * Delete chat-server/trigger.me (#165) (5542f41) * update repo name in code (#163) (f9628bb) * Update README.md (5ad9cd0) * Launch pre releases (#162) (9b2380b) * Expand Contributing Guide and `ingest` README (#161) (30434e3) * filter out very small chunks (#160) (d952c6d) * (DOCSP-31928): Implement numbered lists (#154) (f41258d) * clarify ingest readme (115b00b) * move git install to correct docker stage (40600cb) * (DOCSP-32991, DOCSP-32992): `GitDataSource`, functionality for converting HTML to Markdown, ingest Scala, Java Reactive Streams, and C docs (#152) (7118444) * (DOCSP-32915) [UI] Streamer bugs out when switching tabs (#156) (91c288d) * (DOCSP-33090) [UI] Remove experimental badge from input on focus (#155) (a3ea236) * (DOCSP-32993, DOCSP-32906, DOCSP-32993, DOCSP-32990): Data sources for MD on Github + data ingestion of related content (#148) (7daed63) * change recommended prompt (#150) (2521f08) * add missing env example (#153) (1201a34) * [Snyk] Upgrade mongodb from 5.8.0 to 5.8.1 (#151) (966a4bf) * make preprocessor less biased toward shell (#149) (886cbe5) * (DOCSP-32616): Refactor config, add library, and log config (#143) (4fcfc58) * add default + remove unused style (0296b42) * Update chat-ui release config (3174b61) * Release chat-ui-v0.0.5 (65727f7) * Add root release:chat-ui command (4621989) * (DOCSP-32757) [UI] Update to Responsive LG Chat Components (#145) (81b70bb) * (DOCSP-32670): Basic server load testing (#136) (282c947) * [Snyk] Upgrade urllib from 3.17.2 to 3.18.0 (#147) (aefa9e0) * (DOCSP-32756): Implement headings in tables (#140) (ea416ce) * custom/synthetic/meta data source content (#127) (d687312) * [Snyk] Upgrade urllib from 3.17.1 to 3.17.2 (#144) (209135a) * (DOCSP-32673) [UI] Suppress focus to prevent modal reopen (#139) (43b8a98) * (DOCSP-32936): Increase single IP rate limits to be more line w firewall rate limits (#141) (87d8785) * (DOCSP-31863): Fix flaky test, refactor core llm service + server implementation (#142) (584a5e3) * (DOCSP-32916): Conversation service rate limiting + slow down (#138) (a88a5d5) * (DOCSP-32914): Stop unindenting context (#137) (27630c6) * (DOCSP-31285): Ingest PyMongo docs (#118) (e04c10b) * (DOCSP-32694) Dark mode: use LG palette & provider + cleanup (a95f5c5) * (DOCSP-32694) [UI] Add darkMode prop (#129) (820229c) * (DOCSP-32896) Remove mention of <CONTEXT> to reduce damage of prompt leakage (#135) (1484426) * Update internal docs (#134) (a3b2f56) * Handle Snooty Data API tombstones (#133) (a775da4) * (DOCSP-32867): Update QA DB name for sept 1 release (#131) (ce4fa3f) * (DOCSP-32868): only boost 2 results + order results by score (#130) (5230992) * (DOCSP-32856) Add a toggle for streamed vs awaited responses to the demo (#126) (94e8a20) * (DOCSP-32808): k8s production readiness (#125) (2cdf539) * Upgrade azure openai (#128) (6b85e63) * (DOCSP-32759) [UI] Use text instead of a banner for "validate answers" (#124) (302b3c7) * (DOCSP-32604) More helpful max messages error message (#123) (1be4087) * (DOCSP-32254): Include information about latest product versions + minor prompt engineering (#116) (2e40123) * (DOCSP-32657): Remove same IP address validation on conversations (#112) (731ec20) * (DOCSP-31108): Add prod index scripts (#120) (3efe895) * (DOCSP-32807): Add pm2 for running server (#122) (f659e4b) * [Snyk] Security upgrade mongodb from 5.7.0 to 5.8.0 (#121) (c985fef) * Autoscaling fix (#119) (641199a) * Source GIT_COMMIT env variable (#95) (647b00a) * Allow DOP origins in staging (#117) (38a9b66) * (DOCSP-32074): Handle dev center page metadata (#96) (e283121) * Release chat-ui-v0.0.4 (d3bad60) * (DOCSP-32371): Allow for embedded content re-ingest after changes to chunking algorithm (#114) (8b9d68e) * (DOCSP-32279): Update prompts to not mention context when don't know answer (#113) (71ca44b) * (DOCSP-32688) Stop dereferencing spec (#115) (add0dd8) * (DOCSP-31438): Horizontal pod autoscaling (#110) (3c124d4) * fix registry (d52ff7d) * Release chat-ui-v0.0.3 (f8113b7) * Add chat-ui release-it config (731e3d6) * Switch Chatbot to default export (b5db7f0) * Chat UI v0.0.2 (#111) (7e5d684) * (DOCSP-32659): Make server call its own base url if not given a full URL path (#109) (6cdd026) * Allow users to override server URL via props (#108) (708e614) * (DOCSP-32547): Update prod env to use dedicated Azure deployment (#104) (ac93735) * Prod drone fixes (#107) (4df6432) * Update QA server DB (#105) (a12e7c9) * (DOCSP-32452): Dev Center data source remove `<img>` and `<div>` tags + youtube directive (#93) (c5dac2f) * (DOCSP-32596): Set up CORS on the server (#103) (76c9530) * (DOCSP-32104): Do not serve demo site in prod environment (#101) (5f0ea11) * (DOCSP-32247): Fix preprocessed content not added to DB (#100) (57cc24e) * fix handling of openapi specs (#99) (6f9b0e4) * fix hanging promise in async recursive operation (#98) (c974058) * Drone fix (#97) (e54a272) * (DOCSP-32363) [UI] handle cases where the LLM stops mid code example (#85) (9e26c7f) * (DOCSP-32155) [UI] UX Feedback (#81) (c34b246) * (DOCSP-32243) Handle and test pages with page-level code block select (#94) (f6c3f7e) * (DOCSP-32490) [UI] Link to current commit on GitHub (#90) (64dbd0d) * (DOCSP-32242): Handle tabbed Snooty content (#92) (f87f67d) * Restore 'tags' and use 'tagsIn' only internally (#91) (035f018) * (DOCSP-32227): Improve OpenAPI spec ingestion (#88) (f369218) * (DOCSP-31343): Add system diagram and info for ingest (#86) (6e4402b) * (DOCSP-32362): Improve conversation request logging (#89) (b251e0d) * (DOCSP-32253): Add more semantically relevant product names to metadata (#83) (5869f22) * (DOCSP-32434) [UI] Add error text for non-modal input (#87) (5de9282) * (DOCSP-32206): Strip comments from rst -> md (#84) (611a50d) * (DOCSP-32194): Fix drone and k8s for ingest/chat-server staging (#80) (b61c9bc) * Define engines to remove EBADENGINE warning (#82) (f333ef5) * (DOCSP-31440, DOCSP-32075): Add tags to chunk metadata (#78) (254ce2b) * (DOCSP-32217): Change chunker to chunk based on number of tokens + add max context tokens per message (#79) (e7c1cb6) * (DOCSP-32183 & DOCSP-32226): Automate test creation based on YAML file (#77) (30eb660) * Add database index creation scripts (#75) (53f296f) * (DOCSP-31575): LLM qualitative testing framework (#74) (f222e54) * (DOCSP-31111) [UI] publish react component to npm (#73) (3baf40e) * Refactor `ConversationService` class as `makeConversationService` func (#76) (6cb9a83) * (DOCSP-32070): LLM preprocessing on user queries (#72) (6ff4219) * (DOCSP-31925): Support tables in parsed snooty MD (#67) (539d403) * Reduce ChatGPT API flakiness + faster timeout when flaky (#69) (8938599) * Update Chat UI Packaging (#71) (2e5a0fb) * (DOCSP-32066): Q&A server reduce amount of backoff on embedding (#68) (fe938b0) * (DOCSP-31639): fix timeout err in tests (#70) (741ea29) * Implement page titles and prepending frontmatter to chunks (#64) (cf56a26) * Fix JS error surfaced in the UI (#66) (5e6866a) * bundle LG Chat with component lib (c4fc213) * Modify ui config for pre-release (135d211) * (DOCSP-31154) [UI] Handle timeout & "conversation too long" errors (#65) (1943608) * (DOCSP-31791): Boost manual in the results for short queries + make app config file (#63) (6449993) * (DOCSP-31523) [UI] Replace custom components with LG Chat (#61) (3ee640b) * Fix ingest drone + err handling (#59) (8307750) * Fix code block lang (#62) (cffcbf0) * Get current branch from Snooty Data API plus allow override (#56) (6114e7c) * Clean up pages for embedding content (#60) (45ce277) * (DOCSP-31799): Parse definition lists (and other relevant content inside of `"target"` nodes) (#58) (2d7d1b3) * rename chat-ui package (#57) (dc7794f) * (DOCSP-31574): Create qualitative testing env (#55) (0a1c92a) * Add tests for data deletion (#48) (9b1417a) * Update Snooty Data Source to strip 'index' (#50) (d6b0b2f) * Add ability to override Snooty project baseUrl (#52) (fccc8ca) * Split up tests + add utils (#53) (0dedc6f) * (DOCSP-31436) [Q&A] Investigate & improve streaming rate (#54) (cc90cd0) * (DOCSP-31120) [Q&A] separate further reading reference links from LLM response (#43) (a604d83) * (DOCSP-30863) [Q&A] type check requests (#23) (5f908c0) * Streamline/nitpick/bikeshed the PR template (#51) (28b4e1b) * (DOCSP-31601): Update server base URL (#49) (1c60a0b) * (DOCSP-31573): Update prompts to address bugs (#47) (a366803) * (DOCSP-30602): Ingest service prod pipeline (#42) (8e60755) * Update pull_request_template.md (ab166a7) * Monorepo docker build fix (#46) (1dcef28) * (DOCSP-31284): Integrate prod Snooty Data API (#44) (f80ce51) * (DOCSP-31360) [UI] Conditional Streaming (#45) (34490bb) * update root package-lock (578b06b) * rename chat-ui package (d7b6dcf) * Add shared dev script to monorepo root (67811d1) * Streaming works (#25) (c173ba2) * (DOCSP-30601): Create production release system (#41) (7472c9d) * (DOCSP-31304): Implement 'all' command (#40) (f0c4cf2) * (DOCSP-31201) [UI] Updated Legalese (#39) (39c0f97) * (DOCSP-30584): ingest service CI testing and staging drone pipeline (#38) (d2e8325) * Fix + improve devcenter source tests (#37) (be0a5eb) * Fix URL building for devcenter paths (#36) (4c75175) * Add logic for ingest resumption (#35) (0f650c2) * stricter LLM to reduce halucination risk + change demo page title (#33) (cfb4628) * (DOCSP-30890) [UI] Make component mobile responsive (#29) (66080a7) * Add minor optimization for rate-limited embed requests (#34) (7547e31) * (DOCSP-31121): Add request tracking across logs (#32) (7fdc1e5) * (DOCSP-30988): Implement ingest E2E (#28) (8467989) * (DOCSP-30843): IP Address tracking and validation (#21) (7335672) * Update seed-data script (4f7a2f7) * (DOCSP-31050): Fix CI tests + run tests on PR (#30) (74fe090) * (DOCSP-30953 & DOCSP-30945): Error and edge case handling (#24) (af1b845) * Add PR template to repo (#26) (5231ef0) * Fix tests (#27) (58599ea) * more specific linter targets (18d2f5d) * Move and refactor EmbeddedContent/Store and clean up core services (#19) (43078a2) * separate format from lint (3a817b7) * remove console logs (#22) (20027ce) * Add lists to LGMarkdown (#20) (14cf986) * (DOCSP-30964): Staging build (#18) (1163ff6) * Fix renames (945337e) * Rename "chunks" to "embedded content" (#17) (a18bb8d) * Implement Data Sources (#12) (e9c5784) * Move reusable pieces to chat-core (#14) (3babf35) * (DOCSP-30577): POST `/conversations/:conversationId/messages/:messageId/rating` endpoint (#16) (c659173) * E2E (#15) (c85a2aa) * failed message handling (16acff7) * better prompt validation error handling (6408669) * UI tweaks (f803938) * (DOCSP-30921) [UI] Add prompt validation & error handling (1) (cb98ec7) * (DOCSP-30616): `POST /conversations/:conversationId/messages/` route, no streaming (#13) (8451c08) * Add publishing pipeline (42ab17e) * (DOCSP-30575): POST /conversations/ route (#11) (d609171) * (DOCSP-30619, DOCSP-30758, DOCSP-30722): Add MongoDB integration and conversation/content services (#9) (020a501) * Implement ingest store/database (#8) (5b9f8f6) * Add initial design docs (#1) (31d9069) * (DOCSP-30585): Generate search index script (#10) (458403f) * Merge branch 'demodalize' of github.com:nlarew/docs-chatbot (8d46f37) * better avatars (4c7c712) * tweak card heading (40d1dff) * loading ui (58256da) * (DOCSP-30617): OpenAI LLM service (#7) (d4af1f9) * refactor + loading states (4418f8d) * (DOCSP-30618): Create embeddings service (#6) (c54d123) * mock out Q&A API (9d80c20) * Refactor to use a single chat input (6ac351a) * Animation works except for Card ref (2bc2b41) * Rename & refactor (b5385af) * Send button for all inputs with text (9ca1873) * Move input state up (664b456) * Animation works (b8c2f42) * Use card modal WIP (27c2392) * cleanup (d96bcf0) * Fix bad imports (524f896) * (DOCSP-30576): Stub out Q&A server (#4) (899c0a3) * LGMarkdown (33596cf) * Working on LGMarkdown (897b141) * consistent models with backend (319b2ae) * UI lint pass (d8051d9) * Leafie avatar (ebd660b) * Don't require React Context for conversations (86c90b6) * fix modal style (6ae1500) * Add storybook (73a4ec3) * Modal (cac03d2) * fix banner width (088d858) * Merge pull request #3 from mongodb/DOCSP-30565 (5264da5) * Update .drone.yml (a9e3ba5) * working staging build + updated untested prod (fe16f6c) * remove uncreated env secrets (318c501) * fix broken registry path (a1e1dff) * change release (072100f) * update environment info (ff22f4f) * empty state (a66ce15) * fix value file location (ee4f06b) * fix pipeline (89f9e8b) * Trigger build (408d7dc) * Add (broken) tests (329d9e0) * Trigger build (6cae68f) * Fix context (7d213c0) * update path to docker file (f0edac3) * trigger build (e5e64c0) * more debugging (2dfb919) * Use dependency injection for PageStore/ChunkStore interfaces (89577b8) * Remove start comment (287182d) * move drone file (7453f99) * trigger build? (95579f6) * test deploy pipeline (aab4632) * add cta (db01cdb) * apply prettier (c49d368) * remove unused imports (85f6ab3) * add prettier (c1cf166) * combined props (ae9d66c) * chat state works (e29ee25) * add temp aria-labelledby prop (a0d4695) * Scaffold the ingest CLI (81a5c41) * Update ingest/README (65d2902) * chatbot-ui -> chat-ui (4340d1a) * Merge pull request #2 from mongodben/DOCSP-30486 (17662e6) * Bootstrap ingest CLI tool (0590da2) * Add chatbot-ui (997bec5) * set up QA server project structure (7cd25d5) * Delete specs directory (6b06d68) * ready repo for first push (2b250da) * refactor chatbot server to use latest versions * remove trigger files * clean drone up after changes for development * update contributor guide * Update CONTRIBUTING.md Co-authored-by: Chris Bush <chris.bush@mongodb.com> * Update chat-core/src/extractFrontMatter.ts --------- Co-authored-by: Nick Larew <nick.larew@mongodb.com> Co-authored-by: Chris Bush <chris.bush@mongodb.com> --------- Co-authored-by: Chris Bush <chris.bush@mongodb.com> Co-authored-by: Nick Larew <nick.larew@mongodb.com>
mongodben
added a commit
that referenced
this pull request
Nov 30, 2023
* (EAI-28): Ingest MongoDB University transcript content (#247) (8d7bed9) * Release chat-ui v0.1.2 (c7edbdd) * Upgrade lockfile (90c2a15) * (EAI-145) InputBar adds newlines on shift+enter (#248) (b6f6d84) * (EAI-150) [UI] change "Further Reading" in chatbot response to "Related Resources" (#249) (401771e) * Create sampleMdxFile.mdx (802a614) * (EAI-138): Refactor origin handling, add custom conversation data logic, add conversations middleware (#244) (ce35576) * Deploy docs to https://mongodb.github.io/chatbot/ (#246) (af8ece8) * (EAI-18): Docs for RAG framework (#243) (ef297b2) * Release chat-ui v0.1.1 (f612ffb) * Update exports & format (d159105) * Ubiquitous dark mode (af7c8b8) * Remove unused props & update README (9341493) * Release chat-ui v0.1.0 (8a1853b) * Use a more specific env variable name (f8f4527) * Release chat-ui v0.1.0-0 (65078b7) * Remove @release-it/bumper from chat-ui (9052701) * Remove pending github token (38966a7) * Update pull_request_template.md to use new Jira project (810cb5c) * (EAI-7) [UI] Can't send existing text after reopening modal (#242) (2fbe68f) * (EAI-141): Add link to dev center blog post in README (#241) (6ab06a2) * Initial OSS Refactor (#208) (bba6322) * [Snyk] Upgrade mongodb from 5.8.1 to 5.9.1 (#238) (be8f145) * (EAI-137) [Q&A] Custom request origins (#237) (7d08d4e) * Artifactory auth with token instead of username/password (#239) (a6d1521) * (DOCSP-33030) [Q&A] Record request origin for usage tracking (#234) (eb13f4b) * (EAI-123) [UI] Replace chat-core with mongodb-rag-core (#236) (2afbdce) * (DOCSP-33938) [UI] Fix nested newline spacing (#233) (a879642) * (DOCSP-34007) [UI] Font size & spacing issues on live docs site (#235) (7b4bd3c) * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) (b8541a7) * (DOCSP-32603): Add Terraform data source (#205) (0d64743) * dev QoL scripts (3813ad7) * (DOCSP-33744): update QA server DB name for latest snapshot (#210) (5df2bf0) * (DOCSP-32482): Build out qualitative tests (#201) (0535a89) * Conversations refactor (#199) (5a60a15) * Release ingest-v0.1.2 (#204) (ed34507) * [Snyk] Upgrade urllib from 3.19.1 to 3.19.2 (#203) (743cc01) * Add Rust and Entity Framework Snooty data sources (#202) (ff15139) * [Snyk] Upgrade urllib from 3.19.0 to 3.19.1 (#200) (dea05e0) * Release chat-server v0.1.3 (ac542c8) * Use GITHUB_TOKEN for chat-ui releases (38a9e1a) * [Snyk] Upgrade urllib from 3.18.1 to 3.19.0 (#197) (dd2baaf) * Do not fail query on rephrase fail (#196) (8b7c60a) * Release chat-ui v0.0.8 (03bf7cb) * Bundle new @lg-chat components (2f1f11b) * Release chat-ui v0.0.7 (d082a63) * Unify release-it config (419422e) * Release chat-server v0.1.2 (b0b425c) * [Snyk] Upgrade urllib from 3.18.0 to 3.18.1 (#194) (2c98ca4) * [DOCSP-32950] Create a DevCenter Chatbot Component (#193) (c219d03) * (DOCSP-33445) [ANALYTICS] Scrubbed Views (#195) (8ef2bd7) * (DOCSP-33419) [ANALYTICS] Operational Views (#192) (aae8259) * Release scripts v0.1.2 (fbf2bbf) * Add whether message was deemed "Do not answer" to conversation (#191) (955b722) * (DOCSP-31104) [UI] Add tests to FE services (#190) (d3e2000) * Release ingest v0.1.1 (5b06cd2) * Update package-lock.json (a568d00) * Replace GitHub data source (#188) (747d08e) * Update env vars for analyze script (0a46085) * Add scrubbed message analyzer (#187) (9cef6b8) * (DOCSP-33291) [UI] Add tck tracking to links (#185) (acfb6a1) * Update README.md (a38cbf7) * Release 0.1.1 (8ee27c1) * Fix typo in scripts build (0534458) * Release 0.1.0 (09fa262) * Add release-it to scripts (9ea2e71) * Trigger drone run for scripts service (b1ffcc8) * Fix drone scripts (#186) (9bae8e9) * Cron scrub (#184) (d67dda9) * (DOCSP-33273) Open Source Licensing (#183) (00611bf) * De-flake SnootyProjectsInfo test (#181) (9f7250e) * (DOCSP-32930): Add pymongo metadata (#179) (6694209) * Update release-it git add config to add monorepo stuff (679e447) * Update package-lock in monorepo for release (e7308ef) * Release 0.1.1 (d60d14b) * Update release-it pre-hooks (718b96d) * Run npm audit fix (#178) (c74e1a9) * Remove `seed-content` package (#157) (1b9ac43) * Add scrub message script (#177) (ee2f864) * Remove extra padding around initial input (9dc27ad) * (DOCSP-33094): Save conversation embeddings (#175) (9c6e79d) * Ingest release 0.1.0 (#174) (4928eed) * Release chat-ui-v0.0.6 (2bf2a0e) * (DOCSP-33059) [UI] Fix suggested prompts & other LG Chat tweaks (#171) (cde1b74) * chat server v0.1.0 (#170) (ba69ad8) * new commit for build (56ed01e) * trigger ingest build (4052af9) * clean up spec + remove ip address validation from rate msg (#166) (28cf222) * UI Tweaks (#159) (53b392f) * Delete chat-server/trigger.me (#165) (5542f41) * update repo name in code (#163) (f9628bb) * Update README.md (5ad9cd0) * Launch pre releases (#162) (9b2380b) * Expand Contributing Guide and `ingest` README (#161) (30434e3) * filter out very small chunks (#160) (d952c6d) * (DOCSP-31928): Implement numbered lists (#154) (f41258d) * clarify ingest readme (115b00b) * move git install to correct docker stage (40600cb) * (DOCSP-32991, DOCSP-32992): `GitDataSource`, functionality for converting HTML to Markdown, ingest Scala, Java Reactive Streams, and C docs (#152) (7118444) * (DOCSP-32915) [UI] Streamer bugs out when switching tabs (#156) (91c288d) * (DOCSP-33090) [UI] Remove experimental badge from input on focus (#155) (a3ea236) * (DOCSP-32993, DOCSP-32906, DOCSP-32993, DOCSP-32990): Data sources for MD on Github + data ingestion of related content (#148) (7daed63) * change recommended prompt (#150) (2521f08) * add missing env example (#153) (1201a34) * [Snyk] Upgrade mongodb from 5.8.0 to 5.8.1 (#151) (966a4bf) * make preprocessor less biased toward shell (#149) (886cbe5) * (DOCSP-32616): Refactor config, add library, and log config (#143) (4fcfc58) * add default + remove unused style (0296b42) * Update chat-ui release config (3174b61) * Release chat-ui-v0.0.5 (65727f7) * Add root release:chat-ui command (4621989) * (DOCSP-32757) [UI] Update to Responsive LG Chat Components (#145) (81b70bb) * (DOCSP-32670): Basic server load testing (#136) (282c947) * [Snyk] Upgrade urllib from 3.17.2 to 3.18.0 (#147) (aefa9e0) * (DOCSP-32756): Implement headings in tables (#140) (ea416ce) * custom/synthetic/meta data source content (#127) (d687312) * [Snyk] Upgrade urllib from 3.17.1 to 3.17.2 (#144) (209135a) * (DOCSP-32673) [UI] Suppress focus to prevent modal reopen (#139) (43b8a98) * (DOCSP-32936): Increase single IP rate limits to be more line w firewall rate limits (#141) (87d8785) * (DOCSP-31863): Fix flaky test, refactor core llm service + server implementation (#142) (584a5e3) * (DOCSP-32916): Conversation service rate limiting + slow down (#138) (a88a5d5) * (DOCSP-32914): Stop unindenting context (#137) (27630c6) * (DOCSP-31285): Ingest PyMongo docs (#118) (e04c10b) * (DOCSP-32694) Dark mode: use LG palette & provider + cleanup (a95f5c5) * (DOCSP-32694) [UI] Add darkMode prop (#129) (820229c) * (DOCSP-32896) Remove mention of <CONTEXT> to reduce damage of prompt leakage (#135) (1484426) * Update internal docs (#134) (a3b2f56) * Handle Snooty Data API tombstones (#133) (a775da4) * (DOCSP-32867): Update QA DB name for sept 1 release (#131) (ce4fa3f) * (DOCSP-32868): only boost 2 results + order results by score (#130) (5230992) * (DOCSP-32856) Add a toggle for streamed vs awaited responses to the demo (#126) (94e8a20) * (DOCSP-32808): k8s production readiness (#125) (2cdf539) * Upgrade azure openai (#128) (6b85e63) * (DOCSP-32759) [UI] Use text instead of a banner for "validate answers" (#124) (302b3c7) * (DOCSP-32604) More helpful max messages error message (#123) (1be4087) * (DOCSP-32254): Include information about latest product versions + minor prompt engineering (#116) (2e40123) * (DOCSP-32657): Remove same IP address validation on conversations (#112) (731ec20) * (DOCSP-31108): Add prod index scripts (#120) (3efe895) * (DOCSP-32807): Add pm2 for running server (#122) (f659e4b) * [Snyk] Security upgrade mongodb from 5.7.0 to 5.8.0 (#121) (c985fef) * Autoscaling fix (#119) (641199a) * Source GIT_COMMIT env variable (#95) (647b00a) * Allow DOP origins in staging (#117) (38a9b66) * (DOCSP-32074): Handle dev center page metadata (#96) (e283121) * Release chat-ui-v0.0.4 (d3bad60) * (DOCSP-32371): Allow for embedded content re-ingest after changes to chunking algorithm (#114) (8b9d68e) * (DOCSP-32279): Update prompts to not mention context when don't know answer (#113) (71ca44b) * (DOCSP-32688) Stop dereferencing spec (#115) (add0dd8) * (DOCSP-31438): Horizontal pod autoscaling (#110) (3c124d4) * fix registry (d52ff7d) * Release chat-ui-v0.0.3 (f8113b7) * Add chat-ui release-it config (731e3d6) * Switch Chatbot to default export (b5db7f0) * Chat UI v0.0.2 (#111) (7e5d684) * (DOCSP-32659): Make server call its own base url if not given a full URL path (#109) (6cdd026) * Allow users to override server URL via props (#108) (708e614) * (DOCSP-32547): Update prod env to use dedicated Azure deployment (#104) (ac93735) * Prod drone fixes (#107) (4df6432) * Update QA server DB (#105) (a12e7c9) * (DOCSP-32452): Dev Center data source remove `<img>` and `<div>` tags + youtube directive (#93) (c5dac2f) * (DOCSP-32596): Set up CORS on the server (#103) (76c9530) * (DOCSP-32104): Do not serve demo site in prod environment (#101) (5f0ea11) * (DOCSP-32247): Fix preprocessed content not added to DB (#100) (57cc24e) * fix handling of openapi specs (#99) (6f9b0e4) * fix hanging promise in async recursive operation (#98) (c974058) * Drone fix (#97) (e54a272) * (DOCSP-32363) [UI] handle cases where the LLM stops mid code example (#85) (9e26c7f) * (DOCSP-32155) [UI] UX Feedback (#81) (c34b246) * (DOCSP-32243) Handle and test pages with page-level code block select (#94) (f6c3f7e) * (DOCSP-32490) [UI] Link to current commit on GitHub (#90) (64dbd0d) * (DOCSP-32242): Handle tabbed Snooty content (#92) (f87f67d) * Restore 'tags' and use 'tagsIn' only internally (#91) (035f018) * (DOCSP-32227): Improve OpenAPI spec ingestion (#88) (f369218) * (DOCSP-31343): Add system diagram and info for ingest (#86) (6e4402b) * (DOCSP-32362): Improve conversation request logging (#89) (b251e0d) * (DOCSP-32253): Add more semantically relevant product names to metadata (#83) (5869f22) * (DOCSP-32434) [UI] Add error text for non-modal input (#87) (5de9282) * (DOCSP-32206): Strip comments from rst -> md (#84) (611a50d) * (DOCSP-32194): Fix drone and k8s for ingest/chat-server staging (#80) (b61c9bc) * Define engines to remove EBADENGINE warning (#82) (f333ef5) * (DOCSP-31440, DOCSP-32075): Add tags to chunk metadata (#78) (254ce2b) * (DOCSP-32217): Change chunker to chunk based on number of tokens + add max context tokens per message (#79) (e7c1cb6) * (DOCSP-32183 & DOCSP-32226): Automate test creation based on YAML file (#77) (30eb660) * Add database index creation scripts (#75) (53f296f) * (DOCSP-31575): LLM qualitative testing framework (#74) (f222e54) * (DOCSP-31111) [UI] publish react component to npm (#73) (3baf40e) * Refactor `ConversationService` class as `makeConversationService` func (#76) (6cb9a83) * (DOCSP-32070): LLM preprocessing on user queries (#72) (6ff4219) * (DOCSP-31925): Support tables in parsed snooty MD (#67) (539d403) * Reduce ChatGPT API flakiness + faster timeout when flaky (#69) (8938599) * Update Chat UI Packaging (#71) (2e5a0fb) * (DOCSP-32066): Q&A server reduce amount of backoff on embedding (#68) (fe938b0) * (DOCSP-31639): fix timeout err in tests (#70) (741ea29) * Implement page titles and prepending frontmatter to chunks (#64) (cf56a26) * Fix JS error surfaced in the UI (#66) (5e6866a) * bundle LG Chat with component lib (c4fc213) * Modify ui config for pre-release (135d211) * (DOCSP-31154) [UI] Handle timeout & "conversation too long" errors (#65) (1943608) * (DOCSP-31791): Boost manual in the results for short queries + make app config file (#63) (6449993) * (DOCSP-31523) [UI] Replace custom components with LG Chat (#61) (3ee640b) * Fix ingest drone + err handling (#59) (8307750) * Fix code block lang (#62) (cffcbf0) * Get current branch from Snooty Data API plus allow override (#56) (6114e7c) * Clean up pages for embedding content (#60) (45ce277) * (DOCSP-31799): Parse definition lists (and other relevant content inside of `"target"` nodes) (#58) (2d7d1b3) * rename chat-ui package (#57) (dc7794f) * (DOCSP-31574): Create qualitative testing env (#55) (0a1c92a) * Add tests for data deletion (#48) (9b1417a) * Update Snooty Data Source to strip 'index' (#50) (d6b0b2f) * Add ability to override Snooty project baseUrl (#52) (fccc8ca) * Split up tests + add utils (#53) (0dedc6f) * (DOCSP-31436) [Q&A] Investigate & improve streaming rate (#54) (cc90cd0) * (DOCSP-31120) [Q&A] separate further reading reference links from LLM response (#43) (a604d83) * (DOCSP-30863) [Q&A] type check requests (#23) (5f908c0) * Streamline/nitpick/bikeshed the PR template (#51) (28b4e1b) * (DOCSP-31601): Update server base URL (#49) (1c60a0b) * (DOCSP-31573): Update prompts to address bugs (#47) (a366803) * (DOCSP-30602): Ingest service prod pipeline (#42) (8e60755) * Update pull_request_template.md (ab166a7) * Monorepo docker build fix (#46) (1dcef28) * (DOCSP-31284): Integrate prod Snooty Data API (#44) (f80ce51) * (DOCSP-31360) [UI] Conditional Streaming (#45) (34490bb) * update root package-lock (578b06b) * rename chat-ui package (d7b6dcf) * Add shared dev script to monorepo root (67811d1) * Streaming works (#25) (c173ba2) * (DOCSP-30601): Create production release system (#41) (7472c9d) * (DOCSP-31304): Implement 'all' command (#40) (f0c4cf2) * (DOCSP-31201) [UI] Updated Legalese (#39) (39c0f97) * (DOCSP-30584): ingest service CI testing and staging drone pipeline (#38) (d2e8325) * Fix + improve devcenter source tests (#37) (be0a5eb) * Fix URL building for devcenter paths (#36) (4c75175) * Add logic for ingest resumption (#35) (0f650c2) * stricter LLM to reduce halucination risk + change demo page title (#33) (cfb4628) * (DOCSP-30890) [UI] Make component mobile responsive (#29) (66080a7) * Add minor optimization for rate-limited embed requests (#34) (7547e31) * (DOCSP-31121): Add request tracking across logs (#32) (7fdc1e5) * (DOCSP-30988): Implement ingest E2E (#28) (8467989) * (DOCSP-30843): IP Address tracking and validation (#21) (7335672) * Update seed-data script (4f7a2f7) * (DOCSP-31050): Fix CI tests + run tests on PR (#30) (74fe090) * (DOCSP-30953 & DOCSP-30945): Error and edge case handling (#24) (af1b845) * Add PR template to repo (#26) (5231ef0) * Fix tests (#27) (58599ea) * more specific linter targets (18d2f5d) * Move and refactor EmbeddedContent/Store and clean up core services (#19) (43078a2) * separate format from lint (3a817b7) * remove console logs (#22) (20027ce) * Add lists to LGMarkdown (#20) (14cf986) * (DOCSP-30964): Staging build (#18) (1163ff6) * Fix renames (945337e) * Rename "chunks" to "embedded content" (#17) (a18bb8d) * Implement Data Sources (#12) (e9c5784) * Move reusable pieces to chat-core (#14) (3babf35) * (DOCSP-30577): POST `/conversations/:conversationId/messages/:messageId/rating` endpoint (#16) (c659173) * E2E (#15) (c85a2aa) * failed message handling (16acff7) * better prompt validation error handling (6408669) * UI tweaks (f803938) * (DOCSP-30921) [UI] Add prompt validation & error handling (1) (cb98ec7) * (DOCSP-30616): `POST /conversations/:conversationId/messages/` route, no streaming (#13) (8451c08) * Add publishing pipeline (42ab17e) * (DOCSP-30575): POST /conversations/ route (#11) (d609171) * (DOCSP-30619, DOCSP-30758, DOCSP-30722): Add MongoDB integration and conversation/content services (#9) (020a501) * Implement ingest store/database (#8) (5b9f8f6) * Add initial design docs (#1) (31d9069) * (DOCSP-30585): Generate search index script (#10) (458403f) * Merge branch 'demodalize' of github.com:nlarew/docs-chatbot (8d46f37) * better avatars (4c7c712) * tweak card heading (40d1dff) * loading ui (58256da) * (DOCSP-30617): OpenAI LLM service (#7) (d4af1f9) * refactor + loading states (4418f8d) * (DOCSP-30618): Create embeddings service (#6) (c54d123) * mock out Q&A API (9d80c20) * Refactor to use a single chat input (6ac351a) * Animation works except for Card ref (2bc2b41) * Rename & refactor (b5385af) * Send button for all inputs with text (9ca1873) * Move input state up (664b456) * Animation works (b8c2f42) * Use card modal WIP (27c2392) * cleanup (d96bcf0) * Fix bad imports (524f896) * (DOCSP-30576): Stub out Q&A server (#4) (899c0a3) * LGMarkdown (33596cf) * Working on LGMarkdown (897b141) * consistent models with backend (319b2ae) * UI lint pass (d8051d9) * Leafie avatar (ebd660b) * Don't require React Context for conversations (86c90b6) * fix modal style (6ae1500) * Add storybook (73a4ec3) * Modal (cac03d2) * fix banner width (088d858) * Merge pull request #3 from mongodb/DOCSP-30565 (5264da5) * Update .drone.yml (a9e3ba5) * working staging build + updated untested prod (fe16f6c) * remove uncreated env secrets (318c501) * fix broken registry path (a1e1dff) * change release (072100f) * update environment info (ff22f4f) * empty state (a66ce15) * fix value file location (ee4f06b) * fix pipeline (89f9e8b) * Trigger build (408d7dc) * Add (broken) tests (329d9e0) * Trigger build (6cae68f) * Fix context (7d213c0) * update path to docker file (f0edac3) * trigger build (e5e64c0) * more debugging (2dfb919) * Use dependency injection for PageStore/ChunkStore interfaces (89577b8) * Remove start comment (287182d) * move drone file (7453f99) * trigger build? (95579f6) * test deploy pipeline (aab4632) * add cta (db01cdb) * apply prettier (c49d368) * remove unused imports (85f6ab3) * add prettier (c1cf166) * combined props (ae9d66c) * chat state works (e29ee25) * add temp aria-labelledby prop (a0d4695) * Scaffold the ingest CLI (81a5c41) * Update ingest/README (65d2902) * chatbot-ui -> chat-ui (4340d1a) * Merge pull request #2 from mongodben/DOCSP-30486 (17662e6) * Bootstrap ingest CLI tool (0590da2) * Add chatbot-ui (997bec5) * set up QA server project structure (7cd25d5) * Delete specs directory (6b06d68) * ready repo for first push (2b250da)
mongodben
added a commit
that referenced
this pull request
Nov 30, 2023
* Fix broken CI (#253) (7e44368) * Release mongodb-chatbot-server v0.2.0 (#252) (7dbd308) * (EAI-28): Ingest MongoDB University transcript content (#247) (8d7bed9) * Release chat-ui v0.1.2 (c7edbdd) * Upgrade lockfile (90c2a15) * (EAI-145) InputBar adds newlines on shift+enter (#248) (b6f6d84) * (EAI-150) [UI] change "Further Reading" in chatbot response to "Related Resources" (#249) (401771e) * Create sampleMdxFile.mdx (802a614) * (EAI-138): Refactor origin handling, add custom conversation data logic, add conversations middleware (#244) (ce35576) * Deploy docs to https://mongodb.github.io/chatbot/ (#246) (af8ece8) * (EAI-18): Docs for RAG framework (#243) (ef297b2) * Release chat-ui v0.1.1 (f612ffb) * Update exports & format (d159105) * Ubiquitous dark mode (af7c8b8) * Remove unused props & update README (9341493) * Release chat-ui v0.1.0 (8a1853b) * Use a more specific env variable name (f8f4527) * Release chat-ui v0.1.0-0 (65078b7) * Remove @release-it/bumper from chat-ui (9052701) * Remove pending github token (38966a7) * Update pull_request_template.md to use new Jira project (810cb5c) * (EAI-7) [UI] Can't send existing text after reopening modal (#242) (2fbe68f) * (EAI-141): Add link to dev center blog post in README (#241) (6ab06a2) * Initial OSS Refactor (#208) (bba6322) * [Snyk] Upgrade mongodb from 5.8.1 to 5.9.1 (#238) (be8f145) * (EAI-137) [Q&A] Custom request origins (#237) (7d08d4e) * Artifactory auth with token instead of username/password (#239) (a6d1521) * (DOCSP-33030) [Q&A] Record request origin for usage tracking (#234) (eb13f4b) * (EAI-123) [UI] Replace chat-core with mongodb-rag-core (#236) (2afbdce) * (DOCSP-33938) [UI] Fix nested newline spacing (#233) (a879642) * (DOCSP-34007) [UI] Font size & spacing issues on live docs site (#235) (7b4bd3c) * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) (b8541a7) * (DOCSP-32603): Add Terraform data source (#205) (0d64743) * dev QoL scripts (3813ad7) * (DOCSP-33744): update QA server DB name for latest snapshot (#210) (5df2bf0) * (DOCSP-32482): Build out qualitative tests (#201) (0535a89) * Conversations refactor (#199) (5a60a15) * Release ingest-v0.1.2 (#204) (ed34507) * [Snyk] Upgrade urllib from 3.19.1 to 3.19.2 (#203) (743cc01) * Add Rust and Entity Framework Snooty data sources (#202) (ff15139) * [Snyk] Upgrade urllib from 3.19.0 to 3.19.1 (#200) (dea05e0) * Release chat-server v0.1.3 (ac542c8) * Use GITHUB_TOKEN for chat-ui releases (38a9e1a) * [Snyk] Upgrade urllib from 3.18.1 to 3.19.0 (#197) (dd2baaf) * Do not fail query on rephrase fail (#196) (8b7c60a) * Release chat-ui v0.0.8 (03bf7cb) * Bundle new @lg-chat components (2f1f11b) * Release chat-ui v0.0.7 (d082a63) * Unify release-it config (419422e) * Release chat-server v0.1.2 (b0b425c) * [Snyk] Upgrade urllib from 3.18.0 to 3.18.1 (#194) (2c98ca4) * [DOCSP-32950] Create a DevCenter Chatbot Component (#193) (c219d03) * (DOCSP-33445) [ANALYTICS] Scrubbed Views (#195) (8ef2bd7) * (DOCSP-33419) [ANALYTICS] Operational Views (#192) (aae8259) * Release scripts v0.1.2 (fbf2bbf) * Add whether message was deemed "Do not answer" to conversation (#191) (955b722) * (DOCSP-31104) [UI] Add tests to FE services (#190) (d3e2000) * Release ingest v0.1.1 (5b06cd2) * Update package-lock.json (a568d00) * Replace GitHub data source (#188) (747d08e) * Update env vars for analyze script (0a46085) * Add scrubbed message analyzer (#187) (9cef6b8) * (DOCSP-33291) [UI] Add tck tracking to links (#185) (acfb6a1) * Update README.md (a38cbf7) * Release 0.1.1 (8ee27c1) * Fix typo in scripts build (0534458) * Release 0.1.0 (09fa262) * Add release-it to scripts (9ea2e71) * Trigger drone run for scripts service (b1ffcc8) * Fix drone scripts (#186) (9bae8e9) * Cron scrub (#184) (d67dda9) * (DOCSP-33273) Open Source Licensing (#183) (00611bf) * De-flake SnootyProjectsInfo test (#181) (9f7250e) * (DOCSP-32930): Add pymongo metadata (#179) (6694209) * Update release-it git add config to add monorepo stuff (679e447) * Update package-lock in monorepo for release (e7308ef) * Release 0.1.1 (d60d14b) * Update release-it pre-hooks (718b96d) * Run npm audit fix (#178) (c74e1a9) * Remove `seed-content` package (#157) (1b9ac43) * Add scrub message script (#177) (ee2f864) * Remove extra padding around initial input (9dc27ad) * (DOCSP-33094): Save conversation embeddings (#175) (9c6e79d) * Ingest release 0.1.0 (#174) (4928eed) * Release chat-ui-v0.0.6 (2bf2a0e) * (DOCSP-33059) [UI] Fix suggested prompts & other LG Chat tweaks (#171) (cde1b74) * chat server v0.1.0 (#170) (ba69ad8) * new commit for build (56ed01e) * trigger ingest build (4052af9) * clean up spec + remove ip address validation from rate msg (#166) (28cf222) * UI Tweaks (#159) (53b392f) * Delete chat-server/trigger.me (#165) (5542f41) * update repo name in code (#163) (f9628bb) * Update README.md (5ad9cd0) * Launch pre releases (#162) (9b2380b) * Expand Contributing Guide and `ingest` README (#161) (30434e3) * filter out very small chunks (#160) (d952c6d) * (DOCSP-31928): Implement numbered lists (#154) (f41258d) * clarify ingest readme (115b00b) * move git install to correct docker stage (40600cb) * (DOCSP-32991, DOCSP-32992): `GitDataSource`, functionality for converting HTML to Markdown, ingest Scala, Java Reactive Streams, and C docs (#152) (7118444) * (DOCSP-32915) [UI] Streamer bugs out when switching tabs (#156) (91c288d) * (DOCSP-33090) [UI] Remove experimental badge from input on focus (#155) (a3ea236) * (DOCSP-32993, DOCSP-32906, DOCSP-32993, DOCSP-32990): Data sources for MD on Github + data ingestion of related content (#148) (7daed63) * change recommended prompt (#150) (2521f08) * add missing env example (#153) (1201a34) * [Snyk] Upgrade mongodb from 5.8.0 to 5.8.1 (#151) (966a4bf) * make preprocessor less biased toward shell (#149) (886cbe5) * (DOCSP-32616): Refactor config, add library, and log config (#143) (4fcfc58) * add default + remove unused style (0296b42) * Update chat-ui release config (3174b61) * Release chat-ui-v0.0.5 (65727f7) * Add root release:chat-ui command (4621989) * (DOCSP-32757) [UI] Update to Responsive LG Chat Components (#145) (81b70bb) * (DOCSP-32670): Basic server load testing (#136) (282c947) * [Snyk] Upgrade urllib from 3.17.2 to 3.18.0 (#147) (aefa9e0) * (DOCSP-32756): Implement headings in tables (#140) (ea416ce) * custom/synthetic/meta data source content (#127) (d687312) * [Snyk] Upgrade urllib from 3.17.1 to 3.17.2 (#144) (209135a) * (DOCSP-32673) [UI] Suppress focus to prevent modal reopen (#139) (43b8a98) * (DOCSP-32936): Increase single IP rate limits to be more line w firewall rate limits (#141) (87d8785) * (DOCSP-31863): Fix flaky test, refactor core llm service + server implementation (#142) (584a5e3) * (DOCSP-32916): Conversation service rate limiting + slow down (#138) (a88a5d5) * (DOCSP-32914): Stop unindenting context (#137) (27630c6) * (DOCSP-31285): Ingest PyMongo docs (#118) (e04c10b) * (DOCSP-32694) Dark mode: use LG palette & provider + cleanup (a95f5c5) * (DOCSP-32694) [UI] Add darkMode prop (#129) (820229c) * (DOCSP-32896) Remove mention of <CONTEXT> to reduce damage of prompt leakage (#135) (1484426) * Update internal docs (#134) (a3b2f56) * Handle Snooty Data API tombstones (#133) (a775da4) * (DOCSP-32867): Update QA DB name for sept 1 release (#131) (ce4fa3f) * (DOCSP-32868): only boost 2 results + order results by score (#130) (5230992) * (DOCSP-32856) Add a toggle for streamed vs awaited responses to the demo (#126) (94e8a20) * (DOCSP-32808): k8s production readiness (#125) (2cdf539) * Upgrade azure openai (#128) (6b85e63) * (DOCSP-32759) [UI] Use text instead of a banner for "validate answers" (#124) (302b3c7) * (DOCSP-32604) More helpful max messages error message (#123) (1be4087) * (DOCSP-32254): Include information about latest product versions + minor prompt engineering (#116) (2e40123) * (DOCSP-32657): Remove same IP address validation on conversations (#112) (731ec20) * (DOCSP-31108): Add prod index scripts (#120) (3efe895) * (DOCSP-32807): Add pm2 for running server (#122) (f659e4b) * [Snyk] Security upgrade mongodb from 5.7.0 to 5.8.0 (#121) (c985fef) * Autoscaling fix (#119) (641199a) * Source GIT_COMMIT env variable (#95) (647b00a) * Allow DOP origins in staging (#117) (38a9b66) * (DOCSP-32074): Handle dev center page metadata (#96) (e283121) * Release chat-ui-v0.0.4 (d3bad60) * (DOCSP-32371): Allow for embedded content re-ingest after changes to chunking algorithm (#114) (8b9d68e) * (DOCSP-32279): Update prompts to not mention context when don't know answer (#113) (71ca44b) * (DOCSP-32688) Stop dereferencing spec (#115) (add0dd8) * (DOCSP-31438): Horizontal pod autoscaling (#110) (3c124d4) * fix registry (d52ff7d) * Release chat-ui-v0.0.3 (f8113b7) * Add chat-ui release-it config (731e3d6) * Switch Chatbot to default export (b5db7f0) * Chat UI v0.0.2 (#111) (7e5d684) * (DOCSP-32659): Make server call its own base url if not given a full URL path (#109) (6cdd026) * Allow users to override server URL via props (#108) (708e614) * (DOCSP-32547): Update prod env to use dedicated Azure deployment (#104) (ac93735) * Prod drone fixes (#107) (4df6432) * Update QA server DB (#105) (a12e7c9) * (DOCSP-32452): Dev Center data source remove `<img>` and `<div>` tags + youtube directive (#93) (c5dac2f) * (DOCSP-32596): Set up CORS on the server (#103) (76c9530) * (DOCSP-32104): Do not serve demo site in prod environment (#101) (5f0ea11) * (DOCSP-32247): Fix preprocessed content not added to DB (#100) (57cc24e) * fix handling of openapi specs (#99) (6f9b0e4) * fix hanging promise in async recursive operation (#98) (c974058) * Drone fix (#97) (e54a272) * (DOCSP-32363) [UI] handle cases where the LLM stops mid code example (#85) (9e26c7f) * (DOCSP-32155) [UI] UX Feedback (#81) (c34b246) * (DOCSP-32243) Handle and test pages with page-level code block select (#94) (f6c3f7e) * (DOCSP-32490) [UI] Link to current commit on GitHub (#90) (64dbd0d) * (DOCSP-32242): Handle tabbed Snooty content (#92) (f87f67d) * Restore 'tags' and use 'tagsIn' only internally (#91) (035f018) * (DOCSP-32227): Improve OpenAPI spec ingestion (#88) (f369218) * (DOCSP-31343): Add system diagram and info for ingest (#86) (6e4402b) * (DOCSP-32362): Improve conversation request logging (#89) (b251e0d) * (DOCSP-32253): Add more semantically relevant product names to metadata (#83) (5869f22) * (DOCSP-32434) [UI] Add error text for non-modal input (#87) (5de9282) * (DOCSP-32206): Strip comments from rst -> md (#84) (611a50d) * (DOCSP-32194): Fix drone and k8s for ingest/chat-server staging (#80) (b61c9bc) * Define engines to remove EBADENGINE warning (#82) (f333ef5) * (DOCSP-31440, DOCSP-32075): Add tags to chunk metadata (#78) (254ce2b) * (DOCSP-32217): Change chunker to chunk based on number of tokens + add max context tokens per message (#79) (e7c1cb6) * (DOCSP-32183 & DOCSP-32226): Automate test creation based on YAML file (#77) (30eb660) * Add database index creation scripts (#75) (53f296f) * (DOCSP-31575): LLM qualitative testing framework (#74) (f222e54) * (DOCSP-31111) [UI] publish react component to npm (#73) (3baf40e) * Refactor `ConversationService` class as `makeConversationService` func (#76) (6cb9a83) * (DOCSP-32070): LLM preprocessing on user queries (#72) (6ff4219) * (DOCSP-31925): Support tables in parsed snooty MD (#67) (539d403) * Reduce ChatGPT API flakiness + faster timeout when flaky (#69) (8938599) * Update Chat UI Packaging (#71) (2e5a0fb) * (DOCSP-32066): Q&A server reduce amount of backoff on embedding (#68) (fe938b0) * (DOCSP-31639): fix timeout err in tests (#70) (741ea29) * Implement page titles and prepending frontmatter to chunks (#64) (cf56a26) * Fix JS error surfaced in the UI (#66) (5e6866a) * bundle LG Chat with component lib (c4fc213) * Modify ui config for pre-release (135d211) * (DOCSP-31154) [UI] Handle timeout & "conversation too long" errors (#65) (1943608) * (DOCSP-31791): Boost manual in the results for short queries + make app config file (#63) (6449993) * (DOCSP-31523) [UI] Replace custom components with LG Chat (#61) (3ee640b) * Fix ingest drone + err handling (#59) (8307750) * Fix code block lang (#62) (cffcbf0) * Get current branch from Snooty Data API plus allow override (#56) (6114e7c) * Clean up pages for embedding content (#60) (45ce277) * (DOCSP-31799): Parse definition lists (and other relevant content inside of `"target"` nodes) (#58) (2d7d1b3) * rename chat-ui package (#57) (dc7794f) * (DOCSP-31574): Create qualitative testing env (#55) (0a1c92a) * Add tests for data deletion (#48) (9b1417a) * Update Snooty Data Source to strip 'index' (#50) (d6b0b2f) * Add ability to override Snooty project baseUrl (#52) (fccc8ca) * Split up tests + add utils (#53) (0dedc6f) * (DOCSP-31436) [Q&A] Investigate & improve streaming rate (#54) (cc90cd0) * (DOCSP-31120) [Q&A] separate further reading reference links from LLM response (#43) (a604d83) * (DOCSP-30863) [Q&A] type check requests (#23) (5f908c0) * Streamline/nitpick/bikeshed the PR template (#51) (28b4e1b) * (DOCSP-31601): Update server base URL (#49) (1c60a0b) * (DOCSP-31573): Update prompts to address bugs (#47) (a366803) * (DOCSP-30602): Ingest service prod pipeline (#42) (8e60755) * Update pull_request_template.md (ab166a7) * Monorepo docker build fix (#46) (1dcef28) * (DOCSP-31284): Integrate prod Snooty Data API (#44) (f80ce51) * (DOCSP-31360) [UI] Conditional Streaming (#45) (34490bb) * update root package-lock (578b06b) * rename chat-ui package (d7b6dcf) * Add shared dev script to monorepo root (67811d1) * Streaming works (#25) (c173ba2) * (DOCSP-30601): Create production release system (#41) (7472c9d) * (DOCSP-31304): Implement 'all' command (#40) (f0c4cf2) * (DOCSP-31201) [UI] Updated Legalese (#39) (39c0f97) * (DOCSP-30584): ingest service CI testing and staging drone pipeline (#38) (d2e8325) * Fix + improve devcenter source tests (#37) (be0a5eb) * Fix URL building for devcenter paths (#36) (4c75175) * Add logic for ingest resumption (#35) (0f650c2) * stricter LLM to reduce halucination risk + change demo page title (#33) (cfb4628) * (DOCSP-30890) [UI] Make component mobile responsive (#29) (66080a7) * Add minor optimization for rate-limited embed requests (#34) (7547e31) * (DOCSP-31121): Add request tracking across logs (#32) (7fdc1e5) * (DOCSP-30988): Implement ingest E2E (#28) (8467989) * (DOCSP-30843): IP Address tracking and validation (#21) (7335672) * Update seed-data script (4f7a2f7) * (DOCSP-31050): Fix CI tests + run tests on PR (#30) (74fe090) * (DOCSP-30953 & DOCSP-30945): Error and edge case handling (#24) (af1b845) * Add PR template to repo (#26) (5231ef0) * Fix tests (#27) (58599ea) * more specific linter targets (18d2f5d) * Move and refactor EmbeddedContent/Store and clean up core services (#19) (43078a2) * separate format from lint (3a817b7) * remove console logs (#22) (20027ce) * Add lists to LGMarkdown (#20) (14cf986) * (DOCSP-30964): Staging build (#18) (1163ff6) * Fix renames (945337e) * Rename "chunks" to "embedded content" (#17) (a18bb8d) * Implement Data Sources (#12) (e9c5784) * Move reusable pieces to chat-core (#14) (3babf35) * (DOCSP-30577): POST `/conversations/:conversationId/messages/:messageId/rating` endpoint (#16) (c659173) * E2E (#15) (c85a2aa) * failed message handling (16acff7) * better prompt validation error handling (6408669) * UI tweaks (f803938) * (DOCSP-30921) [UI] Add prompt validation & error handling (1) (cb98ec7) * (DOCSP-30616): `POST /conversations/:conversationId/messages/` route, no streaming (#13) (8451c08) * Add publishing pipeline (42ab17e) * (DOCSP-30575): POST /conversations/ route (#11) (d609171) * (DOCSP-30619, DOCSP-30758, DOCSP-30722): Add MongoDB integration and conversation/content services (#9) (020a501) * Implement ingest store/database (#8) (5b9f8f6) * Add initial design docs (#1) (31d9069) * (DOCSP-30585): Generate search index script (#10) (458403f) * Merge branch 'demodalize' of github.com:nlarew/docs-chatbot (8d46f37) * better avatars (4c7c712) * tweak card heading (40d1dff) * loading ui (58256da) * (DOCSP-30617): OpenAI LLM service (#7) (d4af1f9) * refactor + loading states (4418f8d) * (DOCSP-30618): Create embeddings service (#6) (c54d123) * mock out Q&A API (9d80c20) * Refactor to use a single chat input (6ac351a) * Animation works except for Card ref (2bc2b41) * Rename & refactor (b5385af) * Send button for all inputs with text (9ca1873) * Move input state up (664b456) * Animation works (b8c2f42) * Use card modal WIP (27c2392) * cleanup (d96bcf0) * Fix bad imports (524f896) * (DOCSP-30576): Stub out Q&A server (#4) (899c0a3) * LGMarkdown (33596cf) * Working on LGMarkdown (897b141) * consistent models with backend (319b2ae) * UI lint pass (d8051d9) * Leafie avatar (ebd660b) * Don't require React Context for conversations (86c90b6) * fix modal style (6ae1500) * Add storybook (73a4ec3) * Modal (cac03d2) * fix banner width (088d858) * Merge pull request #3 from mongodb/DOCSP-30565 (5264da5) * Update .drone.yml (a9e3ba5) * working staging build + updated untested prod (fe16f6c) * remove uncreated env secrets (318c501) * fix broken registry path (a1e1dff) * change release (072100f) * update environment info (ff22f4f) * empty state (a66ce15) * fix value file location (ee4f06b) * fix pipeline (89f9e8b) * Trigger build (408d7dc) * Add (broken) tests (329d9e0) * Trigger build (6cae68f) * Fix context (7d213c0) * update path to docker file (f0edac3) * trigger build (e5e64c0) * more debugging (2dfb919) * Use dependency injection for PageStore/ChunkStore interfaces (89577b8) * Remove start comment (287182d) * move drone file (7453f99) * trigger build? (95579f6) * test deploy pipeline (aab4632) * add cta (db01cdb) * apply prettier (c49d368) * remove unused imports (85f6ab3) * add prettier (c1cf166) * combined props (ae9d66c) * chat state works (e29ee25) * add temp aria-labelledby prop (a0d4695) * Scaffold the ingest CLI (81a5c41) * Update ingest/README (65d2902) * chatbot-ui -> chat-ui (4340d1a) * Merge pull request #2 from mongodben/DOCSP-30486 (17662e6) * Bootstrap ingest CLI tool (0590da2) * Add chatbot-ui (997bec5) * set up QA server project structure (7cd25d5) * Delete specs directory (6b06d68) * ready repo for first push (2b250da)
mongodben
added a commit
that referenced
this pull request
Dec 4, 2023
* update package files (05a7c3d) * (EAI-140): Rename directories to match package names (#257) (0519375) * (EAI-49): Add static site endpoint to `mongodb-chatbot-server` + include in implementations (#255) (905c23b) * Add WIP disclaimers + sidebar update to docs (#256) (6ef3833) * (EAI-44): Ingest Prisma docs (#250) (178fcdb) * (EAI-36): WiredTiger data source (#251) (054435b) * Release mongodb-rag-core-v0.1.0 (#254) (0556fa9) * Fix broken CI (#253) (7e44368) * Release mongodb-chatbot-server v0.2.0 (#252) (7dbd308) * (EAI-28): Ingest MongoDB University transcript content (#247) (8d7bed9) * Release chat-ui v0.1.2 (c7edbdd) * Upgrade lockfile (90c2a15) * (EAI-145) InputBar adds newlines on shift+enter (#248) (b6f6d84) * (EAI-150) [UI] change "Further Reading" in chatbot response to "Related Resources" (#249) (401771e) * Create sampleMdxFile.mdx (802a614) * (EAI-138): Refactor origin handling, add custom conversation data logic, add conversations middleware (#244) (ce35576) * Deploy docs to https://mongodb.github.io/chatbot/ (#246) (af8ece8) * (EAI-18): Docs for RAG framework (#243) (ef297b2) * Release chat-ui v0.1.1 (f612ffb) * Update exports & format (d159105) * Ubiquitous dark mode (af7c8b8) * Remove unused props & update README (9341493) * Release chat-ui v0.1.0 (8a1853b) * Use a more specific env variable name (f8f4527) * Release chat-ui v0.1.0-0 (65078b7) * Remove @release-it/bumper from chat-ui (9052701) * Remove pending github token (38966a7) * Update pull_request_template.md to use new Jira project (810cb5c) * (EAI-7) [UI] Can't send existing text after reopening modal (#242) (2fbe68f) * (EAI-141): Add link to dev center blog post in README (#241) (6ab06a2) * Initial OSS Refactor (#208) (bba6322) * [Snyk] Upgrade mongodb from 5.8.1 to 5.9.1 (#238) (be8f145) * (EAI-137) [Q&A] Custom request origins (#237) (7d08d4e) * Artifactory auth with token instead of username/password (#239) (a6d1521) * (DOCSP-33030) [Q&A] Record request origin for usage tracking (#234) (eb13f4b) * (EAI-123) [UI] Replace chat-core with mongodb-rag-core (#236) (2afbdce) * (DOCSP-33938) [UI] Fix nested newline spacing (#233) (a879642) * (DOCSP-34007) [UI] Font size & spacing issues on live docs site (#235) (7b4bd3c) * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) (b8541a7) * (DOCSP-32603): Add Terraform data source (#205) (0d64743) * dev QoL scripts (3813ad7) * (DOCSP-33744): update QA server DB name for latest snapshot (#210) (5df2bf0) * (DOCSP-32482): Build out qualitative tests (#201) (0535a89) * Conversations refactor (#199) (5a60a15) * Release ingest-v0.1.2 (#204) (ed34507) * [Snyk] Upgrade urllib from 3.19.1 to 3.19.2 (#203) (743cc01) * Add Rust and Entity Framework Snooty data sources (#202) (ff15139) * [Snyk] Upgrade urllib from 3.19.0 to 3.19.1 (#200) (dea05e0) * Release chat-server v0.1.3 (ac542c8) * Use GITHUB_TOKEN for chat-ui releases (38a9e1a) * [Snyk] Upgrade urllib from 3.18.1 to 3.19.0 (#197) (dd2baaf) * Do not fail query on rephrase fail (#196) (8b7c60a) * Release chat-ui v0.0.8 (03bf7cb) * Bundle new @lg-chat components (2f1f11b) * Release chat-ui v0.0.7 (d082a63) * Unify release-it config (419422e) * Release chat-server v0.1.2 (b0b425c) * [Snyk] Upgrade urllib from 3.18.0 to 3.18.1 (#194) (2c98ca4) * [DOCSP-32950] Create a DevCenter Chatbot Component (#193) (c219d03) * (DOCSP-33445) [ANALYTICS] Scrubbed Views (#195) (8ef2bd7) * (DOCSP-33419) [ANALYTICS] Operational Views (#192) (aae8259) * Release scripts v0.1.2 (fbf2bbf) * Add whether message was deemed "Do not answer" to conversation (#191) (955b722) * (DOCSP-31104) [UI] Add tests to FE services (#190) (d3e2000) * Release ingest v0.1.1 (5b06cd2) * Update package-lock.json (a568d00) * Replace GitHub data source (#188) (747d08e) * Update env vars for analyze script (0a46085) * Add scrubbed message analyzer (#187) (9cef6b8) * (DOCSP-33291) [UI] Add tck tracking to links (#185) (acfb6a1) * Update README.md (a38cbf7) * Release 0.1.1 (8ee27c1) * Fix typo in scripts build (0534458) * Release 0.1.0 (09fa262) * Add release-it to scripts (9ea2e71) * Trigger drone run for scripts service (b1ffcc8) * Fix drone scripts (#186) (9bae8e9) * Cron scrub (#184) (d67dda9) * (DOCSP-33273) Open Source Licensing (#183) (00611bf) * De-flake SnootyProjectsInfo test (#181) (9f7250e) * (DOCSP-32930): Add pymongo metadata (#179) (6694209) * Update release-it git add config to add monorepo stuff (679e447) * Update package-lock in monorepo for release (e7308ef) * Release 0.1.1 (d60d14b) * Update release-it pre-hooks (718b96d) * Run npm audit fix (#178) (c74e1a9) * Remove `seed-content` package (#157) (1b9ac43) * Add scrub message script (#177) (ee2f864) * Remove extra padding around initial input (9dc27ad) * (DOCSP-33094): Save conversation embeddings (#175) (9c6e79d) * Ingest release 0.1.0 (#174) (4928eed) * Release chat-ui-v0.0.6 (2bf2a0e) * (DOCSP-33059) [UI] Fix suggested prompts & other LG Chat tweaks (#171) (cde1b74) * chat server v0.1.0 (#170) (ba69ad8) * new commit for build (56ed01e) * trigger ingest build (4052af9) * clean up spec + remove ip address validation from rate msg (#166) (28cf222) * UI Tweaks (#159) (53b392f) * Delete chat-server/trigger.me (#165) (5542f41) * update repo name in code (#163) (f9628bb) * Update README.md (5ad9cd0) * Launch pre releases (#162) (9b2380b) * Expand Contributing Guide and `ingest` README (#161) (30434e3) * filter out very small chunks (#160) (d952c6d) * (DOCSP-31928): Implement numbered lists (#154) (f41258d) * clarify ingest readme (115b00b) * move git install to correct docker stage (40600cb) * (DOCSP-32991, DOCSP-32992): `GitDataSource`, functionality for converting HTML to Markdown, ingest Scala, Java Reactive Streams, and C docs (#152) (7118444) * (DOCSP-32915) [UI] Streamer bugs out when switching tabs (#156) (91c288d) * (DOCSP-33090) [UI] Remove experimental badge from input on focus (#155) (a3ea236) * (DOCSP-32993, DOCSP-32906, DOCSP-32993, DOCSP-32990): Data sources for MD on Github + data ingestion of related content (#148) (7daed63) * change recommended prompt (#150) (2521f08) * add missing env example (#153) (1201a34) * [Snyk] Upgrade mongodb from 5.8.0 to 5.8.1 (#151) (966a4bf) * make preprocessor less biased toward shell (#149) (886cbe5) * (DOCSP-32616): Refactor config, add library, and log config (#143) (4fcfc58) * add default + remove unused style (0296b42) * Update chat-ui release config (3174b61) * Release chat-ui-v0.0.5 (65727f7) * Add root release:chat-ui command (4621989) * (DOCSP-32757) [UI] Update to Responsive LG Chat Components (#145) (81b70bb) * (DOCSP-32670): Basic server load testing (#136) (282c947) * [Snyk] Upgrade urllib from 3.17.2 to 3.18.0 (#147) (aefa9e0) * (DOCSP-32756): Implement headings in tables (#140) (ea416ce) * custom/synthetic/meta data source content (#127) (d687312) * [Snyk] Upgrade urllib from 3.17.1 to 3.17.2 (#144) (209135a) * (DOCSP-32673) [UI] Suppress focus to prevent modal reopen (#139) (43b8a98) * (DOCSP-32936): Increase single IP rate limits to be more line w firewall rate limits (#141) (87d8785) * (DOCSP-31863): Fix flaky test, refactor core llm service + server implementation (#142) (584a5e3) * (DOCSP-32916): Conversation service rate limiting + slow down (#138) (a88a5d5) * (DOCSP-32914): Stop unindenting context (#137) (27630c6) * (DOCSP-31285): Ingest PyMongo docs (#118) (e04c10b) * (DOCSP-32694) Dark mode: use LG palette & provider + cleanup (a95f5c5) * (DOCSP-32694) [UI] Add darkMode prop (#129) (820229c) * (DOCSP-32896) Remove mention of <CONTEXT> to reduce damage of prompt leakage (#135) (1484426) * Update internal docs (#134) (a3b2f56) * Handle Snooty Data API tombstones (#133) (a775da4) * (DOCSP-32867): Update QA DB name for sept 1 release (#131) (ce4fa3f) * (DOCSP-32868): only boost 2 results + order results by score (#130) (5230992) * (DOCSP-32856) Add a toggle for streamed vs awaited responses to the demo (#126) (94e8a20) * (DOCSP-32808): k8s production readiness (#125) (2cdf539) * Upgrade azure openai (#128) (6b85e63) * (DOCSP-32759) [UI] Use text instead of a banner for "validate answers" (#124) (302b3c7) * (DOCSP-32604) More helpful max messages error message (#123) (1be4087) * (DOCSP-32254): Include information about latest product versions + minor prompt engineering (#116) (2e40123) * (DOCSP-32657): Remove same IP address validation on conversations (#112) (731ec20) * (DOCSP-31108): Add prod index scripts (#120) (3efe895) * (DOCSP-32807): Add pm2 for running server (#122) (f659e4b) * [Snyk] Security upgrade mongodb from 5.7.0 to 5.8.0 (#121) (c985fef) * Autoscaling fix (#119) (641199a) * Source GIT_COMMIT env variable (#95) (647b00a) * Allow DOP origins in staging (#117) (38a9b66) * (DOCSP-32074): Handle dev center page metadata (#96) (e283121) * Release chat-ui-v0.0.4 (d3bad60) * (DOCSP-32371): Allow for embedded content re-ingest after changes to chunking algorithm (#114) (8b9d68e) * (DOCSP-32279): Update prompts to not mention context when don't know answer (#113) (71ca44b) * (DOCSP-32688) Stop dereferencing spec (#115) (add0dd8) * (DOCSP-31438): Horizontal pod autoscaling (#110) (3c124d4) * fix registry (d52ff7d) * Release chat-ui-v0.0.3 (f8113b7) * Add chat-ui release-it config (731e3d6) * Switch Chatbot to default export (b5db7f0) * Chat UI v0.0.2 (#111) (7e5d684) * (DOCSP-32659): Make server call its own base url if not given a full URL path (#109) (6cdd026) * Allow users to override server URL via props (#108) (708e614) * (DOCSP-32547): Update prod env to use dedicated Azure deployment (#104) (ac93735) * Prod drone fixes (#107) (4df6432) * Update QA server DB (#105) (a12e7c9) * (DOCSP-32452): Dev Center data source remove `<img>` and `<div>` tags + youtube directive (#93) (c5dac2f) * (DOCSP-32596): Set up CORS on the server (#103) (76c9530) * (DOCSP-32104): Do not serve demo site in prod environment (#101) (5f0ea11) * (DOCSP-32247): Fix preprocessed content not added to DB (#100) (57cc24e) * fix handling of openapi specs (#99) (6f9b0e4) * fix hanging promise in async recursive operation (#98) (c974058) * Drone fix (#97) (e54a272) * (DOCSP-32363) [UI] handle cases where the LLM stops mid code example (#85) (9e26c7f) * (DOCSP-32155) [UI] UX Feedback (#81) (c34b246) * (DOCSP-32243) Handle and test pages with page-level code block select (#94) (f6c3f7e) * (DOCSP-32490) [UI] Link to current commit on GitHub (#90) (64dbd0d) * (DOCSP-32242): Handle tabbed Snooty content (#92) (f87f67d) * Restore 'tags' and use 'tagsIn' only internally (#91) (035f018) * (DOCSP-32227): Improve OpenAPI spec ingestion (#88) (f369218) * (DOCSP-31343): Add system diagram and info for ingest (#86) (6e4402b) * (DOCSP-32362): Improve conversation request logging (#89) (b251e0d) * (DOCSP-32253): Add more semantically relevant product names to metadata (#83) (5869f22) * (DOCSP-32434) [UI] Add error text for non-modal input (#87) (5de9282) * (DOCSP-32206): Strip comments from rst -> md (#84) (611a50d) * (DOCSP-32194): Fix drone and k8s for ingest/chat-server staging (#80) (b61c9bc) * Define engines to remove EBADENGINE warning (#82) (f333ef5) * (DOCSP-31440, DOCSP-32075): Add tags to chunk metadata (#78) (254ce2b) * (DOCSP-32217): Change chunker to chunk based on number of tokens + add max context tokens per message (#79) (e7c1cb6) * (DOCSP-32183 & DOCSP-32226): Automate test creation based on YAML file (#77) (30eb660) * Add database index creation scripts (#75) (53f296f) * (DOCSP-31575): LLM qualitative testing framework (#74) (f222e54) * (DOCSP-31111) [UI] publish react component to npm (#73) (3baf40e) * Refactor `ConversationService` class as `makeConversationService` func (#76) (6cb9a83) * (DOCSP-32070): LLM preprocessing on user queries (#72) (6ff4219) * (DOCSP-31925): Support tables in parsed snooty MD (#67) (539d403) * Reduce ChatGPT API flakiness + faster timeout when flaky (#69) (8938599) * Update Chat UI Packaging (#71) (2e5a0fb) * (DOCSP-32066): Q&A server reduce amount of backoff on embedding (#68) (fe938b0) * (DOCSP-31639): fix timeout err in tests (#70) (741ea29) * Implement page titles and prepending frontmatter to chunks (#64) (cf56a26) * Fix JS error surfaced in the UI (#66) (5e6866a) * bundle LG Chat with component lib (c4fc213) * Modify ui config for pre-release (135d211) * (DOCSP-31154) [UI] Handle timeout & "conversation too long" errors (#65) (1943608) * (DOCSP-31791): Boost manual in the results for short queries + make app config file (#63) (6449993) * (DOCSP-31523) [UI] Replace custom components with LG Chat (#61) (3ee640b) * Fix ingest drone + err handling (#59) (8307750) * Fix code block lang (#62) (cffcbf0) * Get current branch from Snooty Data API plus allow override (#56) (6114e7c) * Clean up pages for embedding content (#60) (45ce277) * (DOCSP-31799): Parse definition lists (and other relevant content inside of `"target"` nodes) (#58) (2d7d1b3) * rename chat-ui package (#57) (dc7794f) * (DOCSP-31574): Create qualitative testing env (#55) (0a1c92a) * Add tests for data deletion (#48) (9b1417a) * Update Snooty Data Source to strip 'index' (#50) (d6b0b2f) * Add ability to override Snooty project baseUrl (#52) (fccc8ca) * Split up tests + add utils (#53) (0dedc6f) * (DOCSP-31436) [Q&A] Investigate & improve streaming rate (#54) (cc90cd0) * (DOCSP-31120) [Q&A] separate further reading reference links from LLM response (#43) (a604d83) * (DOCSP-30863) [Q&A] type check requests (#23) (5f908c0) * Streamline/nitpick/bikeshed the PR template (#51) (28b4e1b) * (DOCSP-31601): Update server base URL (#49) (1c60a0b) * (DOCSP-31573): Update prompts to address bugs (#47) (a366803) * (DOCSP-30602): Ingest service prod pipeline (#42) (8e60755) * Update pull_request_template.md (ab166a7) * Monorepo docker build fix (#46) (1dcef28) * (DOCSP-31284): Integrate prod Snooty Data API (#44) (f80ce51) * (DOCSP-31360) [UI] Conditional Streaming (#45) (34490bb) * update root package-lock (578b06b) * rename chat-ui package (d7b6dcf) * Add shared dev script to monorepo root (67811d1) * Streaming works (#25) (c173ba2) * (DOCSP-30601): Create production release system (#41) (7472c9d) * (DOCSP-31304): Implement 'all' command (#40) (f0c4cf2) * (DOCSP-31201) [UI] Updated Legalese (#39) (39c0f97) * (DOCSP-30584): ingest service CI testing and staging drone pipeline (#38) (d2e8325) * Fix + improve devcenter source tests (#37) (be0a5eb) * Fix URL building for devcenter paths (#36) (4c75175) * Add logic for ingest resumption (#35) (0f650c2) * stricter LLM to reduce halucination risk + change demo page title (#33) (cfb4628) * (DOCSP-30890) [UI] Make component mobile responsive (#29) (66080a7) * Add minor optimization for rate-limited embed requests (#34) (7547e31) * (DOCSP-31121): Add request tracking across logs (#32) (7fdc1e5) * (DOCSP-30988): Implement ingest E2E (#28) (8467989) * (DOCSP-30843): IP Address tracking and validation (#21) (7335672) * Update seed-data script (4f7a2f7) * (DOCSP-31050): Fix CI tests + run tests on PR (#30) (74fe090) * (DOCSP-30953 & DOCSP-30945): Error and edge case handling (#24) (af1b845) * Add PR template to repo (#26) (5231ef0) * Fix tests (#27) (58599ea) * more specific linter targets (18d2f5d) * Move and refactor EmbeddedContent/Store and clean up core services (#19) (43078a2) * separate format from lint (3a817b7) * remove console logs (#22) (20027ce) * Add lists to LGMarkdown (#20) (14cf986) * (DOCSP-30964): Staging build (#18) (1163ff6) * Fix renames (945337e) * Rename "chunks" to "embedded content" (#17) (a18bb8d) * Implement Data Sources (#12) (e9c5784) * Move reusable pieces to chat-core (#14) (3babf35) * (DOCSP-30577): POST `/conversations/:conversationId/messages/:messageId/rating` endpoint (#16) (c659173) * E2E (#15) (c85a2aa) * failed message handling (16acff7) * better prompt validation error handling (6408669) * UI tweaks (f803938) * (DOCSP-30921) [UI] Add prompt validation & error handling (1) (cb98ec7) * (DOCSP-30616): `POST /conversations/:conversationId/messages/` route, no streaming (#13) (8451c08) * Add publishing pipeline (42ab17e) * (DOCSP-30575): POST /conversations/ route (#11) (d609171) * (DOCSP-30619, DOCSP-30758, DOCSP-30722): Add MongoDB integration and conversation/content services (#9) (020a501) * Implement ingest store/database (#8) (5b9f8f6) * Add initial design docs (#1) (31d9069) * (DOCSP-30585): Generate search index script (#10) (458403f) * Merge branch 'demodalize' of github.com:nlarew/docs-chatbot (8d46f37) * better avatars (4c7c712) * tweak card heading (40d1dff) * loading ui (58256da) * (DOCSP-30617): OpenAI LLM service (#7) (d4af1f9) * refactor + loading states (4418f8d) * (DOCSP-30618): Create embeddings service (#6) (c54d123) * mock out Q&A API (9d80c20) * Refactor to use a single chat input (6ac351a) * Animation works except for Card ref (2bc2b41) * Rename & refactor (b5385af) * Send button for all inputs with text (9ca1873) * Move input state up (664b456) * Animation works (b8c2f42) * Use card modal WIP (27c2392) * cleanup (d96bcf0) * Fix bad imports (524f896) * (DOCSP-30576): Stub out Q&A server (#4) (899c0a3) * LGMarkdown (33596cf) * Working on LGMarkdown (897b141) * consistent models with backend (319b2ae) * UI lint pass (d8051d9) * Leafie avatar (ebd660b) * Don't require React Context for conversations (86c90b6) * fix modal style (6ae1500) * Add storybook (73a4ec3) * Modal (cac03d2) * fix banner width (088d858) * Merge pull request #3 from mongodb/DOCSP-30565 (5264da5) * Update .drone.yml (a9e3ba5) * working staging build + updated untested prod (fe16f6c) * remove uncreated env secrets (318c501) * fix broken registry path (a1e1dff) * change release (072100f) * update environment info (ff22f4f) * empty state (a66ce15) * fix value file location (ee4f06b) * fix pipeline (89f9e8b) * Trigger build (408d7dc) * Add (broken) tests (329d9e0) * Trigger build (6cae68f) * Fix context (7d213c0) * update path to docker file (f0edac3) * trigger build (e5e64c0) * more debugging (2dfb919) * Use dependency injection for PageStore/ChunkStore interfaces (89577b8) * Remove start comment (287182d) * move drone file (7453f99) * trigger build? (95579f6) * test deploy pipeline (aab4632) * add cta (db01cdb) * apply prettier (c49d368) * remove unused imports (85f6ab3) * add prettier (c1cf166) * combined props (ae9d66c) * chat state works (e29ee25) * add temp aria-labelledby prop (a0d4695) * Scaffold the ingest CLI (81a5c41) * Update ingest/README (65d2902) * chatbot-ui -> chat-ui (4340d1a) * Merge pull request #2 from mongodben/DOCSP-30486 (17662e6) * Bootstrap ingest CLI tool (0590da2) * Add chatbot-ui (997bec5) * set up QA server project structure (7cd25d5) * Delete specs directory (6b06d68) * ready repo for first push (2b250da)
mongodben
added a commit
that referenced
this pull request
Dec 4, 2023
* update package files * Release mongodb-rag-ingest v0.1.7 * update package files (05a7c3d) * (EAI-140): Rename directories to match package names (#257) (0519375) * (EAI-49): Add static site endpoint to `mongodb-chatbot-server` + include in implementations (#255) (905c23b) * Add WIP disclaimers + sidebar update to docs (#256) (6ef3833) * (EAI-44): Ingest Prisma docs (#250) (178fcdb) * (EAI-36): WiredTiger data source (#251) (054435b) * Release mongodb-rag-core-v0.1.0 (#254) (0556fa9) * Fix broken CI (#253) (7e44368) * Release mongodb-chatbot-server v0.2.0 (#252) (7dbd308) * (EAI-28): Ingest MongoDB University transcript content (#247) (8d7bed9) * Release chat-ui v0.1.2 (c7edbdd) * Upgrade lockfile (90c2a15) * (EAI-145) InputBar adds newlines on shift+enter (#248) (b6f6d84) * (EAI-150) [UI] change "Further Reading" in chatbot response to "Related Resources" (#249) (401771e) * Create sampleMdxFile.mdx (802a614) * (EAI-138): Refactor origin handling, add custom conversation data logic, add conversations middleware (#244) (ce35576) * Deploy docs to https://mongodb.github.io/chatbot/ (#246) (af8ece8) * (EAI-18): Docs for RAG framework (#243) (ef297b2) * Release chat-ui v0.1.1 (f612ffb) * Update exports & format (d159105) * Ubiquitous dark mode (af7c8b8) * Remove unused props & update README (9341493) * Release chat-ui v0.1.0 (8a1853b) * Use a more specific env variable name (f8f4527) * Release chat-ui v0.1.0-0 (65078b7) * Remove @release-it/bumper from chat-ui (9052701) * Remove pending github token (38966a7) * Update pull_request_template.md to use new Jira project (810cb5c) * (EAI-7) [UI] Can't send existing text after reopening modal (#242) (2fbe68f) * (EAI-141): Add link to dev center blog post in README (#241) (6ab06a2) * Initial OSS Refactor (#208) (bba6322) * [Snyk] Upgrade mongodb from 5.8.1 to 5.9.1 (#238) (be8f145) * (EAI-137) [Q&A] Custom request origins (#237) (7d08d4e) * Artifactory auth with token instead of username/password (#239) (a6d1521) * (DOCSP-33030) [Q&A] Record request origin for usage tracking (#234) (eb13f4b) * (EAI-123) [UI] Replace chat-core with mongodb-rag-core (#236) (2afbdce) * (DOCSP-33938) [UI] Fix nested newline spacing (#233) (a879642) * (DOCSP-34007) [UI] Font size & spacing issues on live docs site (#235) (7b4bd3c) * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) (b8541a7) * (DOCSP-32603): Add Terraform data source (#205) (0d64743) * dev QoL scripts (3813ad7) * (DOCSP-33744): update QA server DB name for latest snapshot (#210) (5df2bf0) * (DOCSP-32482): Build out qualitative tests (#201) (0535a89) * Conversations refactor (#199) (5a60a15) * Release ingest-v0.1.2 (#204) (ed34507) * [Snyk] Upgrade urllib from 3.19.1 to 3.19.2 (#203) (743cc01) * Add Rust and Entity Framework Snooty data sources (#202) (ff15139) * [Snyk] Upgrade urllib from 3.19.0 to 3.19.1 (#200) (dea05e0) * Release chat-server v0.1.3 (ac542c8) * Use GITHUB_TOKEN for chat-ui releases (38a9e1a) * [Snyk] Upgrade urllib from 3.18.1 to 3.19.0 (#197) (dd2baaf) * Do not fail query on rephrase fail (#196) (8b7c60a) * Release chat-ui v0.0.8 (03bf7cb) * Bundle new @lg-chat components (2f1f11b) * Release chat-ui v0.0.7 (d082a63) * Unify release-it config (419422e) * Release chat-server v0.1.2 (b0b425c) * [Snyk] Upgrade urllib from 3.18.0 to 3.18.1 (#194) (2c98ca4) * [DOCSP-32950] Create a DevCenter Chatbot Component (#193) (c219d03) * (DOCSP-33445) [ANALYTICS] Scrubbed Views (#195) (8ef2bd7) * (DOCSP-33419) [ANALYTICS] Operational Views (#192) (aae8259) * Release scripts v0.1.2 (fbf2bbf) * Add whether message was deemed "Do not answer" to conversation (#191) (955b722) * (DOCSP-31104) [UI] Add tests to FE services (#190) (d3e2000) * Release ingest v0.1.1 (5b06cd2) * Update package-lock.json (a568d00) * Replace GitHub data source (#188) (747d08e) * Update env vars for analyze script (0a46085) * Add scrubbed message analyzer (#187) (9cef6b8) * (DOCSP-33291) [UI] Add tck tracking to links (#185) (acfb6a1) * Update README.md (a38cbf7) * Release 0.1.1 (8ee27c1) * Fix typo in scripts build (0534458) * Release 0.1.0 (09fa262) * Add release-it to scripts (9ea2e71) * Trigger drone run for scripts service (b1ffcc8) * Fix drone scripts (#186) (9bae8e9) * Cron scrub (#184) (d67dda9) * (DOCSP-33273) Open Source Licensing (#183) (00611bf) * De-flake SnootyProjectsInfo test (#181) (9f7250e) * (DOCSP-32930): Add pymongo metadata (#179) (6694209) * Update release-it git add config to add monorepo stuff (679e447) * Update package-lock in monorepo for release (e7308ef) * Release 0.1.1 (d60d14b) * Update release-it pre-hooks (718b96d) * Run npm audit fix (#178) (c74e1a9) * Remove `seed-content` package (#157) (1b9ac43) * Add scrub message script (#177) (ee2f864) * Remove extra padding around initial input (9dc27ad) * (DOCSP-33094): Save conversation embeddings (#175) (9c6e79d) * Ingest release 0.1.0 (#174) (4928eed) * Release chat-ui-v0.0.6 (2bf2a0e) * (DOCSP-33059) [UI] Fix suggested prompts & other LG Chat tweaks (#171) (cde1b74) * chat server v0.1.0 (#170) (ba69ad8) * new commit for build (56ed01e) * trigger ingest build (4052af9) * clean up spec + remove ip address validation from rate msg (#166) (28cf222) * UI Tweaks (#159) (53b392f) * Delete chat-server/trigger.me (#165) (5542f41) * update repo name in code (#163) (f9628bb) * Update README.md (5ad9cd0) * Launch pre releases (#162) (9b2380b) * Expand Contributing Guide and `ingest` README (#161) (30434e3) * filter out very small chunks (#160) (d952c6d) * (DOCSP-31928): Implement numbered lists (#154) (f41258d) * clarify ingest readme (115b00b) * move git install to correct docker stage (40600cb) * (DOCSP-32991, DOCSP-32992): `GitDataSource`, functionality for converting HTML to Markdown, ingest Scala, Java Reactive Streams, and C docs (#152) (7118444) * (DOCSP-32915) [UI] Streamer bugs out when switching tabs (#156) (91c288d) * (DOCSP-33090) [UI] Remove experimental badge from input on focus (#155) (a3ea236) * (DOCSP-32993, DOCSP-32906, DOCSP-32993, DOCSP-32990): Data sources for MD on Github + data ingestion of related content (#148) (7daed63) * change recommended prompt (#150) (2521f08) * add missing env example (#153) (1201a34) * [Snyk] Upgrade mongodb from 5.8.0 to 5.8.1 (#151) (966a4bf) * make preprocessor less biased toward shell (#149) (886cbe5) * (DOCSP-32616): Refactor config, add library, and log config (#143) (4fcfc58) * add default + remove unused style (0296b42) * Update chat-ui release config (3174b61) * Release chat-ui-v0.0.5 (65727f7) * Add root release:chat-ui command (4621989) * (DOCSP-32757) [UI] Update to Responsive LG Chat Components (#145) (81b70bb) * (DOCSP-32670): Basic server load testing (#136) (282c947) * [Snyk] Upgrade urllib from 3.17.2 to 3.18.0 (#147) (aefa9e0) * (DOCSP-32756): Implement headings in tables (#140) (ea416ce) * custom/synthetic/meta data source content (#127) (d687312) * [Snyk] Upgrade urllib from 3.17.1 to 3.17.2 (#144) (209135a) * (DOCSP-32673) [UI] Suppress focus to prevent modal reopen (#139) (43b8a98) * (DOCSP-32936): Increase single IP rate limits to be more line w firewall rate limits (#141) (87d8785) * (DOCSP-31863): Fix flaky test, refactor core llm service + server implementation (#142) (584a5e3) * (DOCSP-32916): Conversation service rate limiting + slow down (#138) (a88a5d5) * (DOCSP-32914): Stop unindenting context (#137) (27630c6) * (DOCSP-31285): Ingest PyMongo docs (#118) (e04c10b) * (DOCSP-32694) Dark mode: use LG palette & provider + cleanup (a95f5c5) * (DOCSP-32694) [UI] Add darkMode prop (#129) (820229c) * (DOCSP-32896) Remove mention of <CONTEXT> to reduce damage of prompt leakage (#135) (1484426) * Update internal docs (#134) (a3b2f56) * Handle Snooty Data API tombstones (#133) (a775da4) * (DOCSP-32867): Update QA DB name for sept 1 release (#131) (ce4fa3f) * (DOCSP-32868): only boost 2 results + order results by score (#130) (5230992) * (DOCSP-32856) Add a toggle for streamed vs awaited responses to the demo (#126) (94e8a20) * (DOCSP-32808): k8s production readiness (#125) (2cdf539) * Upgrade azure openai (#128) (6b85e63) * (DOCSP-32759) [UI] Use text instead of a banner for "validate answers" (#124) (302b3c7) * (DOCSP-32604) More helpful max messages error message (#123) (1be4087) * (DOCSP-32254): Include information about latest product versions + minor prompt engineering (#116) (2e40123) * (DOCSP-32657): Remove same IP address validation on conversations (#112) (731ec20) * (DOCSP-31108): Add prod index scripts (#120) (3efe895) * (DOCSP-32807): Add pm2 for running server (#122) (f659e4b) * [Snyk] Security upgrade mongodb from 5.7.0 to 5.8.0 (#121) (c985fef) * Autoscaling fix (#119) (641199a) * Source GIT_COMMIT env variable (#95) (647b00a) * Allow DOP origins in staging (#117) (38a9b66) * (DOCSP-32074): Handle dev center page metadata (#96) (e283121) * Release chat-ui-v0.0.4 (d3bad60) * (DOCSP-32371): Allow for embedded content re-ingest after changes to chunking algorithm (#114) (8b9d68e) * (DOCSP-32279): Update prompts to not mention context when don't know answer (#113) (71ca44b) * (DOCSP-32688) Stop dereferencing spec (#115) (add0dd8) * (DOCSP-31438): Horizontal pod autoscaling (#110) (3c124d4) * fix registry (d52ff7d) * Release chat-ui-v0.0.3 (f8113b7) * Add chat-ui release-it config (731e3d6) * Switch Chatbot to default export (b5db7f0) * Chat UI v0.0.2 (#111) (7e5d684) * (DOCSP-32659): Make server call its own base url if not given a full URL path (#109) (6cdd026) * Allow users to override server URL via props (#108) (708e614) * (DOCSP-32547): Update prod env to use dedicated Azure deployment (#104) (ac93735) * Prod drone fixes (#107) (4df6432) * Update QA server DB (#105) (a12e7c9) * (DOCSP-32452): Dev Center data source remove `<img>` and `<div>` tags + youtube directive (#93) (c5dac2f) * (DOCSP-32596): Set up CORS on the server (#103) (76c9530) * (DOCSP-32104): Do not serve demo site in prod environment (#101) (5f0ea11) * (DOCSP-32247): Fix preprocessed content not added to DB (#100) (57cc24e) * fix handling of openapi specs (#99) (6f9b0e4) * fix hanging promise in async recursive operation (#98) (c974058) * Drone fix (#97) (e54a272) * (DOCSP-32363) [UI] handle cases where the LLM stops mid code example (#85) (9e26c7f) * (DOCSP-32155) [UI] UX Feedback (#81) (c34b246) * (DOCSP-32243) Handle and test pages with page-level code block select (#94) (f6c3f7e) * (DOCSP-32490) [UI] Link to current commit on GitHub (#90) (64dbd0d) * (DOCSP-32242): Handle tabbed Snooty content (#92) (f87f67d) * Restore 'tags' and use 'tagsIn' only internally (#91) (035f018) * (DOCSP-32227): Improve OpenAPI spec ingestion (#88) (f369218) * (DOCSP-31343): Add system diagram and info for ingest (#86) (6e4402b) * (DOCSP-32362): Improve conversation request logging (#89) (b251e0d) * (DOCSP-32253): Add more semantically relevant product names to metadata (#83) (5869f22) * (DOCSP-32434) [UI] Add error text for non-modal input (#87) (5de9282) * (DOCSP-32206): Strip comments from rst -> md (#84) (611a50d) * (DOCSP-32194): Fix drone and k8s for ingest/chat-server staging (#80) (b61c9bc) * Define engines to remove EBADENGINE warning (#82) (f333ef5) * (DOCSP-31440, DOCSP-32075): Add tags to chunk metadata (#78) (254ce2b) * (DOCSP-32217): Change chunker to chunk based on number of tokens + add max context tokens per message (#79) (e7c1cb6) * (DOCSP-32183 & DOCSP-32226): Automate test creation based on YAML file (#77) (30eb660) * Add database index creation scripts (#75) (53f296f) * (DOCSP-31575): LLM qualitative testing framework (#74) (f222e54) * (DOCSP-31111) [UI] publish react component to npm (#73) (3baf40e) * Refactor `ConversationService` class as `makeConversationService` func (#76) (6cb9a83) * (DOCSP-32070): LLM preprocessing on user queries (#72) (6ff4219) * (DOCSP-31925): Support tables in parsed snooty MD (#67) (539d403) * Reduce ChatGPT API flakiness + faster timeout when flaky (#69) (8938599) * Update Chat UI Packaging (#71) (2e5a0fb) * (DOCSP-32066): Q&A server reduce amount of backoff on embedding (#68) (fe938b0) * (DOCSP-31639): fix timeout err in tests (#70) (741ea29) * Implement page titles and prepending frontmatter to chunks (#64) (cf56a26) * Fix JS error surfaced in the UI (#66) (5e6866a) * bundle LG Chat with component lib (c4fc213) * Modify ui config for pre-release (135d211) * (DOCSP-31154) [UI] Handle timeout & "conversation too long" errors (#65) (1943608) * (DOCSP-31791): Boost manual in the results for short queries + make app config file (#63) (6449993) * (DOCSP-31523) [UI] Replace custom components with LG Chat (#61) (3ee640b) * Fix ingest drone + err handling (#59) (8307750) * Fix code block lang (#62) (cffcbf0) * Get current branch from Snooty Data API plus allow override (#56) (6114e7c) * Clean up pages for embedding content (#60) (45ce277) * (DOCSP-31799): Parse definition lists (and other relevant content inside of `"target"` nodes) (#58) (2d7d1b3) * rename chat-ui package (#57) (dc7794f) * (DOCSP-31574): Create qualitative testing env (#55) (0a1c92a) * Add tests for data deletion (#48) (9b1417a) * Update Snooty Data Source to strip 'index' (#50) (d6b0b2f) * Add ability to override Snooty project baseUrl (#52) (fccc8ca) * Split up tests + add utils (#53) (0dedc6f) * (DOCSP-31436) [Q&A] Investigate & improve streaming rate (#54) (cc90cd0) * (DOCSP-31120) [Q&A] separate further reading reference links from LLM response (#43) (a604d83) * (DOCSP-30863) [Q&A] type check requests (#23) (5f908c0) * Streamline/nitpick/bikeshed the PR template (#51) (28b4e1b) * (DOCSP-31601): Update server base URL (#49) (1c60a0b) * (DOCSP-31573): Update prompts to address bugs (#47) (a366803) * (DOCSP-30602): Ingest service prod pipeline (#42) (8e60755) * Update pull_request_template.md (ab166a7) * Monorepo docker build fix (#46) (1dcef28) * (DOCSP-31284): Integrate prod Snooty Data API (#44) (f80ce51) * (DOCSP-31360) [UI] Conditional Streaming (#45) (34490bb) * update root package-lock (578b06b) * rename chat-ui package (d7b6dcf) * Add shared dev script to monorepo root (67811d1) * Streaming works (#25) (c173ba2) * (DOCSP-30601): Create production release system (#41) (7472c9d) * (DOCSP-31304): Implement 'all' command (#40) (f0c4cf2) * (DOCSP-31201) [UI] Updated Legalese (#39) (39c0f97) * (DOCSP-30584): ingest service CI testing and staging drone pipeline (#38) (d2e8325) * Fix + improve devcenter source tests (#37) (be0a5eb) * Fix URL building for devcenter paths (#36) (4c75175) * Add logic for ingest resumption (#35) (0f650c2) * stricter LLM to reduce halucination risk + change demo page title (#33) (cfb4628) * (DOCSP-30890) [UI] Make component mobile responsive (#29) (66080a7) * Add minor optimization for rate-limited embed requests (#34) (7547e31) * (DOCSP-31121): Add request tracking across logs (#32) (7fdc1e5) * (DOCSP-30988): Implement ingest E2E (#28) (8467989) * (DOCSP-30843): IP Address tracking and validation (#21) (7335672) * Update seed-data script (4f7a2f7) * (DOCSP-31050): Fix CI tests + run tests on PR (#30) (74fe090) * (DOCSP-30953 & DOCSP-30945): Error and edge case handling (#24) (af1b845) * Add PR template to repo (#26) (5231ef0) * Fix tests (#27) (58599ea) * more specific linter targets (18d2f5d) * Move and refactor EmbeddedContent/Store and clean up core services (#19) (43078a2) * separate format from lint (3a817b7) * remove console logs (#22) (20027ce) * Add lists to LGMarkdown (#20) (14cf986) * (DOCSP-30964): Staging build (#18) (1163ff6) * Fix renames (945337e) * Rename "chunks" to "embedded content" (#17) (a18bb8d) * Implement Data Sources (#12) (e9c5784) * Move reusable pieces to chat-core (#14) (3babf35) * (DOCSP-30577): POST `/conversations/:conversationId/messages/:messageId/rating` endpoint (#16) (c659173) * E2E (#15) (c85a2aa) * failed message handling (16acff7) * better prompt validation error handling (6408669) * UI tweaks (f803938) * (DOCSP-30921) [UI] Add prompt validation & error handling (1) (cb98ec7) * (DOCSP-30616): `POST /conversations/:conversationId/messages/` route, no streaming (#13) (8451c08) * Add publishing pipeline (42ab17e) * (DOCSP-30575): POST /conversations/ route (#11) (d609171) * (DOCSP-30619, DOCSP-30758, DOCSP-30722): Add MongoDB integration and conversation/content services (#9) (020a501) * Implement ingest store/database (#8) (5b9f8f6) * Add initial design docs (#1) (31d9069) * (DOCSP-30585): Generate search index script (#10) (458403f) * Merge branch 'demodalize' of github.com:nlarew/docs-chatbot (8d46f37) * better avatars (4c7c712) * tweak card heading (40d1dff) * loading ui (58256da) * (DOCSP-30617): OpenAI LLM service (#7) (d4af1f9) * refactor + loading states (4418f8d) * (DOCSP-30618): Create embeddings service (#6) (c54d123) * mock out Q&A API (9d80c20) * Refactor to use a single chat input (6ac351a) * Animation works except for Card ref (2bc2b41) * Rename & refactor (b5385af) * Send button for all inputs with text (9ca1873) * Move input state up (664b456) * Animation works (b8c2f42) * Use card modal WIP (27c2392) * cleanup (d96bcf0) * Fix bad imports (524f896) * (DOCSP-30576): Stub out Q&A server (#4) (899c0a3) * LGMarkdown (33596cf) * Working on LGMarkdown (897b141) * consistent models with backend (319b2ae) * UI lint pass (d8051d9) * Leafie avatar (ebd660b) * Don't require React Context for conversations (86c90b6) * fix modal style (6ae1500) * Add storybook (73a4ec3) * Modal (cac03d2) * fix banner width (088d858) * Merge pull request #3 from mongodb/DOCSP-30565 (5264da5) * Update .drone.yml (a9e3ba5) * working staging build + updated untested prod (fe16f6c) * remove uncreated env secrets (318c501) * fix broken registry path (a1e1dff) * change release (072100f) * update environment info (ff22f4f) * empty state (a66ce15) * fix value file location (ee4f06b) * fix pipeline (89f9e8b) * Trigger build (408d7dc) * Add (broken) tests (329d9e0) * Trigger build (6cae68f) * Fix context (7d213c0) * update path to docker file (f0edac3) * trigger build (e5e64c0) * more debugging (2dfb919) * Use dependency injection for PageStore/ChunkStore interfaces (89577b8) * Remove start comment (287182d) * move drone file (7453f99) * trigger build? (95579f6) * test deploy pipeline (aab4632) * add cta (db01cdb) * apply prettier (c49d368) * remove unused imports (85f6ab3) * add prettier (c1cf166) * combined props (ae9d66c) * chat state works (e29ee25) * add temp aria-labelledby prop (a0d4695) * Scaffold the ingest CLI (81a5c41) * Update ingest/README (65d2902) * chatbot-ui -> chat-ui (4340d1a) * Merge pull request #2 from mongodben/DOCSP-30486 (17662e6) * Bootstrap ingest CLI tool (0590da2) * Add chatbot-ui (997bec5) * set up QA server project structure (7cd25d5) * Delete specs directory (6b06d68) * ready repo for first push (2b250da)
mongodben
added a commit
that referenced
this pull request
Dec 4, 2023
* (EAI-140): Rename directories to match package names (#257) (0519375) * (EAI-49): Add static site endpoint to `mongodb-chatbot-server` + include in implementations (#255) (905c23b) * Add WIP disclaimers + sidebar update to docs (#256) (6ef3833) * (EAI-44): Ingest Prisma docs (#250) (178fcdb) * (EAI-36): WiredTiger data source (#251) (054435b) * Release mongodb-rag-core-v0.1.0 (#254) (0556fa9) * Fix broken CI (#253) (7e44368) * Release mongodb-chatbot-server v0.2.0 (#252) (7dbd308) * (EAI-28): Ingest MongoDB University transcript content (#247) (8d7bed9) * Release chat-ui v0.1.2 (c7edbdd) * Upgrade lockfile (90c2a15) * (EAI-145) InputBar adds newlines on shift+enter (#248) (b6f6d84) * (EAI-150) [UI] change "Further Reading" in chatbot response to "Related Resources" (#249) (401771e) * Create sampleMdxFile.mdx (802a614) * (EAI-138): Refactor origin handling, add custom conversation data logic, add conversations middleware (#244) (ce35576) * Deploy docs to https://mongodb.github.io/chatbot/ (#246) (af8ece8) * (EAI-18): Docs for RAG framework (#243) (ef297b2) * Release chat-ui v0.1.1 (f612ffb) * Update exports & format (d159105) * Ubiquitous dark mode (af7c8b8) * Remove unused props & update README (9341493) * Release chat-ui v0.1.0 (8a1853b) * Use a more specific env variable name (f8f4527) * Release chat-ui v0.1.0-0 (65078b7) * Remove @release-it/bumper from chat-ui (9052701) * Remove pending github token (38966a7) * Update pull_request_template.md to use new Jira project (810cb5c) * (EAI-7) [UI] Can't send existing text after reopening modal (#242) (2fbe68f) * (EAI-141): Add link to dev center blog post in README (#241) (6ab06a2) * Initial OSS Refactor (#208) (bba6322) * [Snyk] Upgrade mongodb from 5.8.1 to 5.9.1 (#238) (be8f145) * (EAI-137) [Q&A] Custom request origins (#237) (7d08d4e) * Artifactory auth with token instead of username/password (#239) (a6d1521) * (DOCSP-33030) [Q&A] Record request origin for usage tracking (#234) (eb13f4b) * (EAI-123) [UI] Replace chat-core with mongodb-rag-core (#236) (2afbdce) * (DOCSP-33938) [UI] Fix nested newline spacing (#233) (a879642) * (DOCSP-34007) [UI] Font size & spacing issues on live docs site (#235) (7b4bd3c) * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) (b8541a7) * (DOCSP-32603): Add Terraform data source (#205) (0d64743) * dev QoL scripts (3813ad7) * (DOCSP-33744): update QA server DB name for latest snapshot (#210) (5df2bf0) * (DOCSP-32482): Build out qualitative tests (#201) (0535a89) * Conversations refactor (#199) (5a60a15) * Release ingest-v0.1.2 (#204) (ed34507) * [Snyk] Upgrade urllib from 3.19.1 to 3.19.2 (#203) (743cc01) * Add Rust and Entity Framework Snooty data sources (#202) (ff15139) * [Snyk] Upgrade urllib from 3.19.0 to 3.19.1 (#200) (dea05e0) * Release chat-server v0.1.3 (ac542c8) * Use GITHUB_TOKEN for chat-ui releases (38a9e1a) * [Snyk] Upgrade urllib from 3.18.1 to 3.19.0 (#197) (dd2baaf) * Do not fail query on rephrase fail (#196) (8b7c60a) * Release chat-ui v0.0.8 (03bf7cb) * Bundle new @lg-chat components (2f1f11b) * Release chat-ui v0.0.7 (d082a63) * Unify release-it config (419422e) * Release chat-server v0.1.2 (b0b425c) * [Snyk] Upgrade urllib from 3.18.0 to 3.18.1 (#194) (2c98ca4) * [DOCSP-32950] Create a DevCenter Chatbot Component (#193) (c219d03) * (DOCSP-33445) [ANALYTICS] Scrubbed Views (#195) (8ef2bd7) * (DOCSP-33419) [ANALYTICS] Operational Views (#192) (aae8259) * Release scripts v0.1.2 (fbf2bbf) * Add whether message was deemed "Do not answer" to conversation (#191) (955b722) * (DOCSP-31104) [UI] Add tests to FE services (#190) (d3e2000) * Release ingest v0.1.1 (5b06cd2) * Update package-lock.json (a568d00) * Replace GitHub data source (#188) (747d08e) * Update env vars for analyze script (0a46085) * Add scrubbed message analyzer (#187) (9cef6b8) * (DOCSP-33291) [UI] Add tck tracking to links (#185) (acfb6a1) * Update README.md (a38cbf7) * Release 0.1.1 (8ee27c1) * Fix typo in scripts build (0534458) * Release 0.1.0 (09fa262) * Add release-it to scripts (9ea2e71) * Trigger drone run for scripts service (b1ffcc8) * Fix drone scripts (#186) (9bae8e9) * Cron scrub (#184) (d67dda9) * (DOCSP-33273) Open Source Licensing (#183) (00611bf) * De-flake SnootyProjectsInfo test (#181) (9f7250e) * (DOCSP-32930): Add pymongo metadata (#179) (6694209) * Update release-it git add config to add monorepo stuff (679e447) * Update package-lock in monorepo for release (e7308ef) * Release 0.1.1 (d60d14b) * Update release-it pre-hooks (718b96d) * Run npm audit fix (#178) (c74e1a9) * Remove `seed-content` package (#157) (1b9ac43) * Add scrub message script (#177) (ee2f864) * Remove extra padding around initial input (9dc27ad) * (DOCSP-33094): Save conversation embeddings (#175) (9c6e79d) * Ingest release 0.1.0 (#174) (4928eed) * Release chat-ui-v0.0.6 (2bf2a0e) * (DOCSP-33059) [UI] Fix suggested prompts & other LG Chat tweaks (#171) (cde1b74) * chat server v0.1.0 (#170) (ba69ad8) * new commit for build (56ed01e) * trigger ingest build (4052af9) * clean up spec + remove ip address validation from rate msg (#166) (28cf222) * UI Tweaks (#159) (53b392f) * Delete chat-server/trigger.me (#165) (5542f41) * update repo name in code (#163) (f9628bb) * Update README.md (5ad9cd0) * Launch pre releases (#162) (9b2380b) * Expand Contributing Guide and `ingest` README (#161) (30434e3) * filter out very small chunks (#160) (d952c6d) * (DOCSP-31928): Implement numbered lists (#154) (f41258d) * clarify ingest readme (115b00b) * move git install to correct docker stage (40600cb) * (DOCSP-32991, DOCSP-32992): `GitDataSource`, functionality for converting HTML to Markdown, ingest Scala, Java Reactive Streams, and C docs (#152) (7118444) * (DOCSP-32915) [UI] Streamer bugs out when switching tabs (#156) (91c288d) * (DOCSP-33090) [UI] Remove experimental badge from input on focus (#155) (a3ea236) * (DOCSP-32993, DOCSP-32906, DOCSP-32993, DOCSP-32990): Data sources for MD on Github + data ingestion of related content (#148) (7daed63) * change recommended prompt (#150) (2521f08) * add missing env example (#153) (1201a34) * [Snyk] Upgrade mongodb from 5.8.0 to 5.8.1 (#151) (966a4bf) * make preprocessor less biased toward shell (#149) (886cbe5) * (DOCSP-32616): Refactor config, add library, and log config (#143) (4fcfc58) * add default + remove unused style (0296b42) * Update chat-ui release config (3174b61) * Release chat-ui-v0.0.5 (65727f7) * Add root release:chat-ui command (4621989) * (DOCSP-32757) [UI] Update to Responsive LG Chat Components (#145) (81b70bb) * (DOCSP-32670): Basic server load testing (#136) (282c947) * [Snyk] Upgrade urllib from 3.17.2 to 3.18.0 (#147) (aefa9e0) * (DOCSP-32756): Implement headings in tables (#140) (ea416ce) * custom/synthetic/meta data source content (#127) (d687312) * [Snyk] Upgrade urllib from 3.17.1 to 3.17.2 (#144) (209135a) * (DOCSP-32673) [UI] Suppress focus to prevent modal reopen (#139) (43b8a98) * (DOCSP-32936): Increase single IP rate limits to be more line w firewall rate limits (#141) (87d8785) * (DOCSP-31863): Fix flaky test, refactor core llm service + server implementation (#142) (584a5e3) * (DOCSP-32916): Conversation service rate limiting + slow down (#138) (a88a5d5) * (DOCSP-32914): Stop unindenting context (#137) (27630c6) * (DOCSP-31285): Ingest PyMongo docs (#118) (e04c10b) * (DOCSP-32694) Dark mode: use LG palette & provider + cleanup (a95f5c5) * (DOCSP-32694) [UI] Add darkMode prop (#129) (820229c) * (DOCSP-32896) Remove mention of <CONTEXT> to reduce damage of prompt leakage (#135) (1484426) * Update internal docs (#134) (a3b2f56) * Handle Snooty Data API tombstones (#133) (a775da4) * (DOCSP-32867): Update QA DB name for sept 1 release (#131) (ce4fa3f) * (DOCSP-32868): only boost 2 results + order results by score (#130) (5230992) * (DOCSP-32856) Add a toggle for streamed vs awaited responses to the demo (#126) (94e8a20) * (DOCSP-32808): k8s production readiness (#125) (2cdf539) * Upgrade azure openai (#128) (6b85e63) * (DOCSP-32759) [UI] Use text instead of a banner for "validate answers" (#124) (302b3c7) * (DOCSP-32604) More helpful max messages error message (#123) (1be4087) * (DOCSP-32254): Include information about latest product versions + minor prompt engineering (#116) (2e40123) * (DOCSP-32657): Remove same IP address validation on conversations (#112) (731ec20) * (DOCSP-31108): Add prod index scripts (#120) (3efe895) * (DOCSP-32807): Add pm2 for running server (#122) (f659e4b) * [Snyk] Security upgrade mongodb from 5.7.0 to 5.8.0 (#121) (c985fef) * Autoscaling fix (#119) (641199a) * Source GIT_COMMIT env variable (#95) (647b00a) * Allow DOP origins in staging (#117) (38a9b66) * (DOCSP-32074): Handle dev center page metadata (#96) (e283121) * Release chat-ui-v0.0.4 (d3bad60) * (DOCSP-32371): Allow for embedded content re-ingest after changes to chunking algorithm (#114) (8b9d68e) * (DOCSP-32279): Update prompts to not mention context when don't know answer (#113) (71ca44b) * (DOCSP-32688) Stop dereferencing spec (#115) (add0dd8) * (DOCSP-31438): Horizontal pod autoscaling (#110) (3c124d4) * fix registry (d52ff7d) * Release chat-ui-v0.0.3 (f8113b7) * Add chat-ui release-it config (731e3d6) * Switch Chatbot to default export (b5db7f0) * Chat UI v0.0.2 (#111) (7e5d684) * (DOCSP-32659): Make server call its own base url if not given a full URL path (#109) (6cdd026) * Allow users to override server URL via props (#108) (708e614) * (DOCSP-32547): Update prod env to use dedicated Azure deployment (#104) (ac93735) * Prod drone fixes (#107) (4df6432) * Update QA server DB (#105) (a12e7c9) * (DOCSP-32452): Dev Center data source remove `<img>` and `<div>` tags + youtube directive (#93) (c5dac2f) * (DOCSP-32596): Set up CORS on the server (#103) (76c9530) * (DOCSP-32104): Do not serve demo site in prod environment (#101) (5f0ea11) * (DOCSP-32247): Fix preprocessed content not added to DB (#100) (57cc24e) * fix handling of openapi specs (#99) (6f9b0e4) * fix hanging promise in async recursive operation (#98) (c974058) * Drone fix (#97) (e54a272) * (DOCSP-32363) [UI] handle cases where the LLM stops mid code example (#85) (9e26c7f) * (DOCSP-32155) [UI] UX Feedback (#81) (c34b246) * (DOCSP-32243) Handle and test pages with page-level code block select (#94) (f6c3f7e) * (DOCSP-32490) [UI] Link to current commit on GitHub (#90) (64dbd0d) * (DOCSP-32242): Handle tabbed Snooty content (#92) (f87f67d) * Restore 'tags' and use 'tagsIn' only internally (#91) (035f018) * (DOCSP-32227): Improve OpenAPI spec ingestion (#88) (f369218) * (DOCSP-31343): Add system diagram and info for ingest (#86) (6e4402b) * (DOCSP-32362): Improve conversation request logging (#89) (b251e0d) * (DOCSP-32253): Add more semantically relevant product names to metadata (#83) (5869f22) * (DOCSP-32434) [UI] Add error text for non-modal input (#87) (5de9282) * (DOCSP-32206): Strip comments from rst -> md (#84) (611a50d) * (DOCSP-32194): Fix drone and k8s for ingest/chat-server staging (#80) (b61c9bc) * Define engines to remove EBADENGINE warning (#82) (f333ef5) * (DOCSP-31440, DOCSP-32075): Add tags to chunk metadata (#78) (254ce2b) * (DOCSP-32217): Change chunker to chunk based on number of tokens + add max context tokens per message (#79) (e7c1cb6) * (DOCSP-32183 & DOCSP-32226): Automate test creation based on YAML file (#77) (30eb660) * Add database index creation scripts (#75) (53f296f) * (DOCSP-31575): LLM qualitative testing framework (#74) (f222e54) * (DOCSP-31111) [UI] publish react component to npm (#73) (3baf40e) * Refactor `ConversationService` class as `makeConversationService` func (#76) (6cb9a83) * (DOCSP-32070): LLM preprocessing on user queries (#72) (6ff4219) * (DOCSP-31925): Support tables in parsed snooty MD (#67) (539d403) * Reduce ChatGPT API flakiness + faster timeout when flaky (#69) (8938599) * Update Chat UI Packaging (#71) (2e5a0fb) * (DOCSP-32066): Q&A server reduce amount of backoff on embedding (#68) (fe938b0) * (DOCSP-31639): fix timeout err in tests (#70) (741ea29) * Implement page titles and prepending frontmatter to chunks (#64) (cf56a26) * Fix JS error surfaced in the UI (#66) (5e6866a) * bundle LG Chat with component lib (c4fc213) * Modify ui config for pre-release (135d211) * (DOCSP-31154) [UI] Handle timeout & "conversation too long" errors (#65) (1943608) * (DOCSP-31791): Boost manual in the results for short queries + make app config file (#63) (6449993) * (DOCSP-31523) [UI] Replace custom components with LG Chat (#61) (3ee640b) * Fix ingest drone + err handling (#59) (8307750) * Fix code block lang (#62) (cffcbf0) * Get current branch from Snooty Data API plus allow override (#56) (6114e7c) * Clean up pages for embedding content (#60) (45ce277) * (DOCSP-31799): Parse definition lists (and other relevant content inside of `"target"` nodes) (#58) (2d7d1b3) * rename chat-ui package (#57) (dc7794f) * (DOCSP-31574): Create qualitative testing env (#55) (0a1c92a) * Add tests for data deletion (#48) (9b1417a) * Update Snooty Data Source to strip 'index' (#50) (d6b0b2f) * Add ability to override Snooty project baseUrl (#52) (fccc8ca) * Split up tests + add utils (#53) (0dedc6f) * (DOCSP-31436) [Q&A] Investigate & improve streaming rate (#54) (cc90cd0) * (DOCSP-31120) [Q&A] separate further reading reference links from LLM response (#43) (a604d83) * (DOCSP-30863) [Q&A] type check requests (#23) (5f908c0) * Streamline/nitpick/bikeshed the PR template (#51) (28b4e1b) * (DOCSP-31601): Update server base URL (#49) (1c60a0b) * (DOCSP-31573): Update prompts to address bugs (#47) (a366803) * (DOCSP-30602): Ingest service prod pipeline (#42) (8e60755) * Update pull_request_template.md (ab166a7) * Monorepo docker build fix (#46) (1dcef28) * (DOCSP-31284): Integrate prod Snooty Data API (#44) (f80ce51) * (DOCSP-31360) [UI] Conditional Streaming (#45) (34490bb) * update root package-lock (578b06b) * rename chat-ui package (d7b6dcf) * Add shared dev script to monorepo root (67811d1) * Streaming works (#25) (c173ba2) * (DOCSP-30601): Create production release system (#41) (7472c9d) * (DOCSP-31304): Implement 'all' command (#40) (f0c4cf2) * (DOCSP-31201) [UI] Updated Legalese (#39) (39c0f97) * (DOCSP-30584): ingest service CI testing and staging drone pipeline (#38) (d2e8325) * Fix + improve devcenter source tests (#37) (be0a5eb) * Fix URL building for devcenter paths (#36) (4c75175) * Add logic for ingest resumption (#35) (0f650c2) * stricter LLM to reduce halucination risk + change demo page title (#33) (cfb4628) * (DOCSP-30890) [UI] Make component mobile responsive (#29) (66080a7) * Add minor optimization for rate-limited embed requests (#34) (7547e31) * (DOCSP-31121): Add request tracking across logs (#32) (7fdc1e5) * (DOCSP-30988): Implement ingest E2E (#28) (8467989) * (DOCSP-30843): IP Address tracking and validation (#21) (7335672) * Update seed-data script (4f7a2f7) * (DOCSP-31050): Fix CI tests + run tests on PR (#30) (74fe090) * (DOCSP-30953 & DOCSP-30945): Error and edge case handling (#24) (af1b845) * Add PR template to repo (#26) (5231ef0) * Fix tests (#27) (58599ea) * more specific linter targets (18d2f5d) * Move and refactor EmbeddedContent/Store and clean up core services (#19) (43078a2) * separate format from lint (3a817b7) * remove console logs (#22) (20027ce) * Add lists to LGMarkdown (#20) (14cf986) * (DOCSP-30964): Staging build (#18) (1163ff6) * Fix renames (945337e) * Rename "chunks" to "embedded content" (#17) (a18bb8d) * Implement Data Sources (#12) (e9c5784) * Move reusable pieces to chat-core (#14) (3babf35) * (DOCSP-30577): POST `/conversations/:conversationId/messages/:messageId/rating` endpoint (#16) (c659173) * E2E (#15) (c85a2aa) * failed message handling (16acff7) * better prompt validation error handling (6408669) * UI tweaks (f803938) * (DOCSP-30921) [UI] Add prompt validation & error handling (1) (cb98ec7) * (DOCSP-30616): `POST /conversations/:conversationId/messages/` route, no streaming (#13) (8451c08) * Add publishing pipeline (42ab17e) * (DOCSP-30575): POST /conversations/ route (#11) (d609171) * (DOCSP-30619, DOCSP-30758, DOCSP-30722): Add MongoDB integration and conversation/content services (#9) (020a501) * Implement ingest store/database (#8) (5b9f8f6) * Add initial design docs (#1) (31d9069) * (DOCSP-30585): Generate search index script (#10) (458403f) * Merge branch 'demodalize' of github.com:nlarew/docs-chatbot (8d46f37) * better avatars (4c7c712) * tweak card heading (40d1dff) * loading ui (58256da) * (DOCSP-30617): OpenAI LLM service (#7) (d4af1f9) * refactor + loading states (4418f8d) * (DOCSP-30618): Create embeddings service (#6) (c54d123) * mock out Q&A API (9d80c20) * Refactor to use a single chat input (6ac351a) * Animation works except for Card ref (2bc2b41) * Rename & refactor (b5385af) * Send button for all inputs with text (9ca1873) * Move input state up (664b456) * Animation works (b8c2f42) * Use card modal WIP (27c2392) * cleanup (d96bcf0) * Fix bad imports (524f896) * (DOCSP-30576): Stub out Q&A server (#4) (899c0a3) * LGMarkdown (33596cf) * Working on LGMarkdown (897b141) * consistent models with backend (319b2ae) * UI lint pass (d8051d9) * Leafie avatar (ebd660b) * Don't require React Context for conversations (86c90b6) * fix modal style (6ae1500) * Add storybook (73a4ec3) * Modal (cac03d2) * fix banner width (088d858) * Merge pull request #3 from mongodb/DOCSP-30565 (5264da5) * Update .drone.yml (a9e3ba5) * working staging build + updated untested prod (fe16f6c) * remove uncreated env secrets (318c501) * fix broken registry path (a1e1dff) * change release (072100f) * update environment info (ff22f4f) * empty state (a66ce15) * fix value file location (ee4f06b) * fix pipeline (89f9e8b) * Trigger build (408d7dc) * Add (broken) tests (329d9e0) * Trigger build (6cae68f) * Fix context (7d213c0) * update path to docker file (f0edac3) * trigger build (e5e64c0) * more debugging (2dfb919) * Use dependency injection for PageStore/ChunkStore interfaces (89577b8) * Remove start comment (287182d) * move drone file (7453f99) * trigger build? (95579f6) * test deploy pipeline (aab4632) * add cta (db01cdb) * apply prettier (c49d368) * remove unused imports (85f6ab3) * add prettier (c1cf166) * combined props (ae9d66c) * chat state works (e29ee25) * add temp aria-labelledby prop (a0d4695) * Scaffold the ingest CLI (81a5c41) * Update ingest/README (65d2902) * chatbot-ui -> chat-ui (4340d1a) * Merge pull request #2 from mongodben/DOCSP-30486 (17662e6) * Bootstrap ingest CLI tool (0590da2) * Add chatbot-ui (997bec5) * set up QA server project structure (7cd25d5) * Delete specs directory (6b06d68) * ready repo for first push (2b250da)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This started as the E2E implementation of the ingest tool, but got too big just merging pieces into the core library and refactoring.