Drone fix #97

mongodben · 2023-08-23T15:32:01Z

Jira: n/a

Changes

Fix issue where ingest service was trying to build when it shouldn't. Before it would sometimes try to deploy something that doesn't exist, causing stuff to break in kubernetes
reformatted drone file to make trigger actions clearer by moving filters from when to trigger

Notes

* Implement page titles and prepending frontmatter to chunks (#64) * Implement page titles and prepending frontmatter to chunks * Implement updating front matter + tests * Leave title undefined if not specified by snooty data * (DOCSP-31639): fix timeout err in tests (#70) fix test err * (DOCSP-32066): Q&A server reduce amount of backoff on embedding (#68) reduce amount of backoff on embedding * Update Chat UI Packaging (#71) * Switch css to Emotion * Reduce ChatGPT API flakiness + faster timeout when flaky (#69) * reduce flakiness + faster timeout when flaky * implement CB feedback * (DOCSP-31925): Support tables in parsed snooty MD (#67) * handle tables as HTML tables * start tests * nested tables * cleanup * Fix list parsing within tables Use a state enum rather than bool for parent table * Update ingest/src/snootyAstToMd.ts Co-authored-by: Nick Larew <nick.larew@10gen.com> * Implement multiple header rows (??) --------- Co-authored-by: Chris Bush <chris.bush@mongodb.com> Co-authored-by: Chris Bush <chris.bush@10gen.com> Co-authored-by: Nick Larew <nick.larew@10gen.com> * (DOCSP-32070): LLM preprocessing on user queries (#72) * make llm preprocessor * Add todo * add 1 word expansion * add frontmatterUpdater() to chat-core * append metadata to vector search query * hook up preprocessor to chat * test non-processing negative response * trigger rebuild * clean up * frontmatter -> frontMatter * rename * rename * implement CB feedback * edits * Add more context * small fixes * Refactor `ConversationService` class as `makeConversationService` func (#76) refactor conversation service as make func * (DOCSP-31111) [UI] publish react component to npm (#73) * (DOCSP-31575): LLM qualitative testing framework (#74) * start stubbing * add check response quality func * works in editor, but doesn't compile * runs but doesn't look correct in editor * working in VSCode and compiling * all working * working again with separate jest suite * remove console log * remove typechat from top lvl * Add back to tsconfig * remove await * Fixes * add test using framework * comment out skipped tests to appease linter * Add database index creation scripts (#75) * (DOCSP-31106) [DEL] Set up TTL on user conversations * (DOCSP-31622) [INGEST] Indexes for embedded_content and pages collections * (DOCSP-32183 & DOCSP-32226): Automate test creation based on YAML file (#77) * Add yaml tests * hypothesis -> name * correct path to yaml file * create scripts project * script to create test YAML files * remove build dir * (DOCSP-32217): Change chunker to chunk based on number of tokens + add max context tokens per message (#79) * add max context tokens per message * Trigger * trigger * strip frontmatter before sending docs to LLM * refactor removeFrontMatter to use package * Refactor includeChunksForMaxTokensPossible() per CB feedback to use findIndex * (DOCSP-31440, DOCSP-32075): Add tags to chunk metadata (#78) * (DOCSP-31440): Add tags to chunk metadata * (DOCSP-32075): Replace 'tags' with 'metadata' in embedded content document - This probably has no practical effect, but would allow for more flexibility in future vector searching or chunk filtering. * Fix seed-data for new format * Update * Remove explicit fetch import * Define engines to remove EBADENGINE warning (#82) * (DOCSP-32194): Fix drone and k8s for ingest/chat-server staging (#80) * Rebuild services * Trigger staging deploy * Trigger ingest build * fix build err * clean up drone file + PR * (DOCSP-32206): Strip comments from rst -> md (#84) * (DOCSP-32434) [UI] Add error text for non-modal input (#87) * (DOCSP-32253): Add more semantically relevant product names to metadata (#83) * Rebuild services * Trigger staging deploy * Trigger ingest build * clean up drone file + PR * refactor with Page.metadata * add test for arbitrary metadata * update EmbeddedContent description * update MongoDbUserQueryPreprocessorResponse for greater semantic meaning * update pre-processor tests * implement review feedback * fix broken tests * Fix lint err * (DOCSP-32362): Improve conversation request logging (#89) improve request logging * (DOCSP-31343): Add system diagram and info for ingest (#86) * Add ingest system diagram * Move diagram to README and add some info * (DOCSP-32227): Improve OpenAPI spec ingestion (#88) * checkpoint * integrate changes w project * clean up + add todo * Draft atlas spec handling * make func async for network call * finish tests * fix broken tests * implement cb feedback + merge fixes * Restore 'tags' and use 'tagsIn' only internally (#91) * (DOCSP-32242): Handle tabbed Snooty content (#92) * add tabs to page + handle chunking * add more table-based delimiters * CB feedback + clean up * (DOCSP-32490) [UI] Link to current commit on GitHub (#90) * (DOCSP-32243) Handle and test pages with page-level code block select (#94) * handle and test pages with page-level codeblock select * clean up comment * add spacing around tables * refactor per CB feedback * Refactor to remove undefined case which should never occur * (DOCSP-32155) [UI] UX Feedback (#81) * (DOCSP-32363) [UI] handle cases where the LLM stops mid code example (#85) * Drone fix (#97) * update drone file for testing * add trigger * fix and trigger * fix and trigger * trigger chat server build * cean up tmp changes * remove triggers * fix handing promise * fix hanging promise in async recursive operation (#98) fix handing promise * fix handling of openapi specs (#99) * (DOCSP-32247): Fix preprocessed content not added to DB (#100) fix preprocessed content not added to DB * (DOCSP-32104): Do not serve demo site in prod environment (#101) only serve staging site if env not prod * (DOCSP-32596): Set up CORS on the server (#103) * draft CORS setup * add tests * (DOCSP-32452): Dev Center data source remove `<img>` and `<div>` tags + youtube directive (#93) dev center data source remove img and div tags + youtube directive * Update QA server DB (#105) update QA server DB --------- Co-authored-by: Chris Bush <chris.bush@mongodb.com> Co-authored-by: Nick Larew <nick.larew@mongodb.com> Co-authored-by: Chris Bush <chris.bush@10gen.com> Co-authored-by: Nick Larew <nick.larew@10gen.com>

* update lock (e45350d) * update core version (7d15d35) * update release conf (05a6686) * remove unused dep (12c73a3) * remove test from release-it b/c covered in CI (99f2a9b) * update drone (932c9a8) * Release mongodb-rag-ingest v0.1.6 (d9e3ea9) * update lock (bd718e6) * update lock (dac1773) * update dependent packages (50a43db) * Release mongodb-rag-core v0.0.5 (55195cb) * Release mongodb-rag-core v0.0.5-3 (b765423) * remove unused bundled dep (87f6f7e) * make ingest reexport core (5b0fd8e) * Add missing typechat dep (4bf2594) * Release mongodb-rag-ingest v0.1.6-2 (a22fbdf) * update lock file (def3cb7) * bunlde chat-core in ingest (3f763fe) * Release mongodb-rag-ingest v0.1.6-1 (fdd536c) * fix drone file (9b16f20) * Release mongodb-rag-ingest v0.1.6-0 (767efac) * update drone publication pipelines (3d761a6) * Release mongodb-rag-core v0.0.5-2 (a5689a2) * fix drone script (f711595) * Release mongodb-rag-core v0.0.5-1 (0a3745a) * Release mongodb-rag-core v0.0.5-0 (580e8ff) * update npm chat-core publish config (5be4922) * update chatbot ui out destination (d3e20a2) * fix build err from test (af0cce0) * update chat-core dependency (069fb9b) * Trigger staging build of mongodb server (a438a82) * update chat-server.dockerfile (81999da) * update executable path (5b06c12) * update mongodb ingest to use latest ingest CLI (b21441f) * Release mongodb-rag-ingest v0.1.5 (bf852d1) * update lock (70a8c65) * make CLI entry point executable (00b57d3) * trigger build of ingest-mongodb-public to test drone (96b4206) * update package-lock (ff83848) * update package-lock.json w new deps (f4d6579) * update dependencies to latest in mongodb ingest (0d6e48a) * Release mongodb-rag-ingest v0.1.4 (7b7e8de) * update monorepo chatcore version (80ebd61) * update ingest version of core to latest working (5a7b346) * Release mongodb-rag-core v0.0.4 (e04b754) * Add simple readme to core (729f44a) * Release mongodb-rag-core v0.0.3 (230ab97) * update package.json files with files to include in release (e4567ed) * Release mongodb-rag-core v0.0.2 (e5e6ac1) * make ingest a node executable (da4011f) * simplify dockerfile (e2d8751) * fix build commands (37d746b) * retry drone (bbbdd82) * trigger ingest build again (1315732) * hack drone to trigger builds (88e3255) * trigger ingest staging build (33c7f34) * Release mongodb-rag-core v0.0.2-8 (2258731) * add publish env vars (1ed87ad) * Release mongodb-rag-core v0.0.2-7 (c5914d3) * fix build issues (965a9e0) * fix flaky tests (9dc7153) * fix broken server tests (e4b04b9) * make ingest tests pass (00c9065) * remove chatbot-server-mongodb-public tests from prod build (d95c984) * remove chat-server tests from prod build (4c0cd22) * working chat server (0248a11) * update core version in server (d236f91) * update package files to use (7d3e5b8) * Release mongodb-rag-core v0.0.2-6 (40688a7) * update pub flow to speed up testing (8e0409c) * add new npm creds (cf6084a) * update drone names (47dd13c) * remove release-it from monorepo conf (ff4434c) * separate chat server package and implementation + update drone file (4441cbf) * update core + ingest release conf (5559403) * Release mongodb-rag-ingest v0.1.3 (9a7a2f9) * add non prerelease version (756d1a7) * update deps (7f4ac98) * update ingest conf (88c47ee) * Release mongodb-rag-core v0.0.2-5 (ba238e6) * add .drone pipelines to deploy new npm packages (f4a8210) * configure so that publishes package from creation of gh tag via drone (f9df609) * Release mongodb-rag-core v0.0.2-4 (d626d91) * remove from / to from github (881e474) * update release conf (0477bc2) * inside (should see) (7f19587) * add outside commit (shouldn't see) (feccfe7) * inside (c22271b) * all (90b568a) * Release mongodb-rag-core v0.0.2-3 (a90c1ba) * Add changelog (4a9af69) * release notes/changelog (c9b3b0f) * add changelog setup (3e90095) * Release mongodb-rag-core v0.0.2-2 (bcf3c9c) * Release mongodb-rag-core v0.0.1 (a3a1355) * update version (f6e1d8f) * add release-it to chat-core (629de3e) * update release conf (73308a6) * update contributing guide (11ee87b) * update repo to use lerna versioning (f7c7285) * Merge remote-tracking branch 'upstream/main' into oss-refactor-pt-2 (412da6a) * Ingest plugins (#211) (0206bee) * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) (b8541a7) * OSS Chat Server (#206) (820e249) * (DOCSP-32603): Add Terraform data source (#205) (0d64743) * dev QoL scripts (3813ad7) * (DOCSP-33744): update QA server DB name for latest snapshot (#210) (5df2bf0) * (DOCSP-32482): Build out qualitative tests (#201) (0535a89) * Organize some files (#207) (f8cdd3c) * Conversations refactor (#199) (5a60a15) * Release ingest-v0.1.2 (#204) (ed34507) * [Snyk] Upgrade urllib from 3.19.1 to 3.19.2 (#203) (743cc01) * Add Rust and Entity Framework Snooty data sources (#202) (ff15139) * [Snyk] Upgrade urllib from 3.19.0 to 3.19.1 (#200) (dea05e0) * Release chat-server v0.1.3 (ac542c8) * Use GITHUB_TOKEN for chat-ui releases (38a9e1a) * [Snyk] Upgrade urllib from 3.18.1 to 3.19.0 (#197) (dd2baaf) * Do not fail query on rephrase fail (#196) (8b7c60a) * Release chat-ui v0.0.8 (03bf7cb) * Bundle new @lg-chat components (2f1f11b) * Release chat-ui v0.0.7 (d082a63) * Unify release-it config (419422e) * Release chat-server v0.1.2 (b0b425c) * [Snyk] Upgrade urllib from 3.18.0 to 3.18.1 (#194) (2c98ca4) * [DOCSP-32950] Create a DevCenter Chatbot Component (#193) (c219d03) * (DOCSP-33445) [ANALYTICS] Scrubbed Views (#195) (8ef2bd7) * (DOCSP-33419) [ANALYTICS] Operational Views (#192) (aae8259) * Release scripts v0.1.2 (fbf2bbf) * Add whether message was deemed "Do not answer" to conversation (#191) (955b722) * (DOCSP-31104) [UI] Add tests to FE services (#190) (d3e2000) * Release ingest v0.1.1 (5b06cd2) * Update package-lock.json (a568d00) * Replace GitHub data source (#188) (747d08e) * Update env vars for analyze script (0a46085) * Add scrubbed message analyzer (#187) (9cef6b8) * (DOCSP-33291) [UI] Add tck tracking to links (#185) (acfb6a1) * Update README.md (a38cbf7) * Release 0.1.1 (8ee27c1) * Fix typo in scripts build (0534458) * Release 0.1.0 (09fa262) * Add release-it to scripts (9ea2e71) * Trigger drone run for scripts service (b1ffcc8) * Fix drone scripts (#186) (9bae8e9) * Cron scrub (#184) (d67dda9) * (DOCSP-33273) Open Source Licensing (#183) (00611bf) * De-flake SnootyProjectsInfo test (#181) (9f7250e) * (DOCSP-32930): Add pymongo metadata (#179) (6694209) * Update release-it git add config to add monorepo stuff (679e447) * Update package-lock in monorepo for release (e7308ef) * Release 0.1.1 (d60d14b) * Update release-it pre-hooks (718b96d) * Run npm audit fix (#178) (c74e1a9) * Remove `seed-content` package (#157) (1b9ac43) * Add scrub message script (#177) (ee2f864) * Remove extra padding around initial input (9dc27ad) * (DOCSP-33094): Save conversation embeddings (#175) (9c6e79d) * Ingest release 0.1.0 (#174) (4928eed) * Release chat-ui-v0.0.6 (2bf2a0e) * (DOCSP-33059) [UI] Fix suggested prompts & other LG Chat tweaks (#171) (cde1b74) * chat server v0.1.0 (#170) (ba69ad8) * new commit for build (56ed01e) * trigger ingest build (4052af9) * clean up spec + remove ip address validation from rate msg (#166) (28cf222) * UI Tweaks (#159) (53b392f) * Delete chat-server/trigger.me (#165) (5542f41) * update repo name in code (#163) (f9628bb) * Update README.md (5ad9cd0) * Launch pre releases (#162) (9b2380b) * Expand Contributing Guide and `ingest` README (#161) (30434e3) * filter out very small chunks (#160) (d952c6d) * (DOCSP-31928): Implement numbered lists (#154) (f41258d) * clarify ingest readme (115b00b) * move git install to correct docker stage (40600cb) * (DOCSP-32991, DOCSP-32992): `GitDataSource`, functionality for converting HTML to Markdown, ingest Scala, Java Reactive Streams, and C docs (#152) (7118444) * (DOCSP-32915) [UI] Streamer bugs out when switching tabs (#156) (91c288d) * (DOCSP-33090) [UI] Remove experimental badge from input on focus (#155) (a3ea236) * (DOCSP-32993, DOCSP-32906, DOCSP-32993, DOCSP-32990): Data sources for MD on Github + data ingestion of related content (#148) (7daed63) * change recommended prompt (#150) (2521f08) * add missing env example (#153) (1201a34) * [Snyk] Upgrade mongodb from 5.8.0 to 5.8.1 (#151) (966a4bf) * make preprocessor less biased toward shell (#149) (886cbe5) * (DOCSP-32616): Refactor config, add library, and log config (#143) (4fcfc58) * add default + remove unused style (0296b42) * Update chat-ui release config (3174b61) * Release chat-ui-v0.0.5 (65727f7) * Add root release:chat-ui command (4621989) * (DOCSP-32757) [UI] Update to Responsive LG Chat Components (#145) (81b70bb) * (DOCSP-32670): Basic server load testing (#136) (282c947) * [Snyk] Upgrade urllib from 3.17.2 to 3.18.0 (#147) (aefa9e0) * (DOCSP-32756): Implement headings in tables (#140) (ea416ce) * custom/synthetic/meta data source content (#127) (d687312) * [Snyk] Upgrade urllib from 3.17.1 to 3.17.2 (#144) (209135a) * (DOCSP-32673) [UI] Suppress focus to prevent modal reopen (#139) (43b8a98) * (DOCSP-32936): Increase single IP rate limits to be more line w firewall rate limits (#141) (87d8785) * (DOCSP-31863): Fix flaky test, refactor core llm service + server implementation (#142) (584a5e3) * (DOCSP-32916): Conversation service rate limiting + slow down (#138) (a88a5d5) * (DOCSP-32914): Stop unindenting context (#137) (27630c6) * (DOCSP-31285): Ingest PyMongo docs (#118) (e04c10b) * (DOCSP-32694) Dark mode: use LG palette & provider + cleanup (a95f5c5) * (DOCSP-32694) [UI] Add darkMode prop (#129) (820229c) * (DOCSP-32896) Remove mention of <CONTEXT> to reduce damage of prompt leakage (#135) (1484426) * Update internal docs (#134) (a3b2f56) * Handle Snooty Data API tombstones (#133) (a775da4) * (DOCSP-32867): Update QA DB name for sept 1 release (#131) (ce4fa3f) * (DOCSP-32868): only boost 2 results + order results by score (#130) (5230992) * (DOCSP-32856) Add a toggle for streamed vs awaited responses to the demo (#126) (94e8a20) * (DOCSP-32808): k8s production readiness (#125) (2cdf539) * Upgrade azure openai (#128) (6b85e63) * (DOCSP-32759) [UI] Use text instead of a banner for "validate answers" (#124) (302b3c7) * (DOCSP-32604) More helpful max messages error message (#123) (1be4087) * (DOCSP-32254): Include information about latest product versions + minor prompt engineering (#116) (2e40123) * (DOCSP-32657): Remove same IP address validation on conversations (#112) (731ec20) * (DOCSP-31108): Add prod index scripts (#120) (3efe895) * (DOCSP-32807): Add pm2 for running server (#122) (f659e4b) * [Snyk] Security upgrade mongodb from 5.7.0 to 5.8.0 (#121) (c985fef) * Autoscaling fix (#119) (641199a) * Source GIT_COMMIT env variable (#95) (647b00a) * Allow DOP origins in staging (#117) (38a9b66) * (DOCSP-32074): Handle dev center page metadata (#96) (e283121) * Release chat-ui-v0.0.4 (d3bad60) * (DOCSP-32371): Allow for embedded content re-ingest after changes to chunking algorithm (#114) (8b9d68e) * (DOCSP-32279): Update prompts to not mention context when don't know answer (#113) (71ca44b) * (DOCSP-32688) Stop dereferencing spec (#115) (add0dd8) * (DOCSP-31438): Horizontal pod autoscaling (#110) (3c124d4) * fix registry (d52ff7d) * Release chat-ui-v0.0.3 (f8113b7) * Add chat-ui release-it config (731e3d6) * Switch Chatbot to default export (b5db7f0) * Chat UI v0.0.2 (#111) (7e5d684) * (DOCSP-32659): Make server call its own base url if not given a full URL path (#109) (6cdd026) * Allow users to override server URL via props (#108) (708e614) * (DOCSP-32547): Update prod env to use dedicated Azure deployment (#104) (ac93735) * Prod drone fixes (#107) (4df6432) * Update QA server DB (#105) (a12e7c9) * (DOCSP-32452): Dev Center data source remove `<img>` and `<div>` tags + youtube directive (#93) (c5dac2f) * (DOCSP-32596): Set up CORS on the server (#103) (76c9530) * (DOCSP-32104): Do not serve demo site in prod environment (#101) (5f0ea11) * (DOCSP-32247): Fix preprocessed content not added to DB (#100) (57cc24e) * fix handling of openapi specs (#99) (6f9b0e4) * fix hanging promise in async recursive operation (#98) (c974058) * Drone fix (#97) (e54a272) * (DOCSP-32363) [UI] handle cases where the LLM stops mid code example (#85) (9e26c7f) * (DOCSP-32155) [UI] UX Feedback (#81) (c34b246) * (DOCSP-32243) Handle and test pages with page-level code block select (#94) (f6c3f7e) * (DOCSP-32490) [UI] Link to current commit on GitHub (#90) (64dbd0d) * (DOCSP-32242): Handle tabbed Snooty content (#92) (f87f67d) * Restore 'tags' and use 'tagsIn' only internally (#91) (035f018) * (DOCSP-32227): Improve OpenAPI spec ingestion (#88) (f369218) * (DOCSP-31343): Add system diagram and info for ingest (#86) (6e4402b) * (DOCSP-32362): Improve conversation request logging (#89) (b251e0d) * (DOCSP-32253): Add more semantically relevant product names to metadata (#83) (5869f22) * (DOCSP-32434) [UI] Add error text for non-modal input (#87) (5de9282) * (DOCSP-32206): Strip comments from rst -> md (#84) (611a50d) * (DOCSP-32194): Fix drone and k8s for ingest/chat-server staging (#80) (b61c9bc) * Define engines to remove EBADENGINE warning (#82) (f333ef5) * (DOCSP-31440, DOCSP-32075): Add tags to chunk metadata (#78) (254ce2b) * (DOCSP-32217): Change chunker to chunk based on number of tokens + add max context tokens per message (#79) (e7c1cb6) * (DOCSP-32183 & DOCSP-32226): Automate test creation based on YAML file (#77) (30eb660) * Add database index creation scripts (#75) (53f296f) * (DOCSP-31575): LLM qualitative testing framework (#74) (f222e54) * (DOCSP-31111) [UI] publish react component to npm (#73) (3baf40e) * Refactor `ConversationService` class as `makeConversationService` func (#76) (6cb9a83) * (DOCSP-32070): LLM preprocessing on user queries (#72) (6ff4219) * (DOCSP-31925): Support tables in parsed snooty MD (#67) (539d403) * Reduce ChatGPT API flakiness + faster timeout when flaky (#69) (8938599) * Update Chat UI Packaging (#71) (2e5a0fb) * (DOCSP-32066): Q&A server reduce amount of backoff on embedding (#68) (fe938b0) * (DOCSP-31639): fix timeout err in tests (#70) (741ea29) * Implement page titles and prepending frontmatter to chunks (#64) (cf56a26) * Fix JS error surfaced in the UI (#66) (5e6866a) * bundle LG Chat with component lib (c4fc213) * Modify ui config for pre-release (135d211) * (DOCSP-31154) [UI] Handle timeout & "conversation too long" errors (#65) (1943608) * (DOCSP-31791): Boost manual in the results for short queries + make app config file (#63) (6449993) * (DOCSP-31523) [UI] Replace custom components with LG Chat (#61) (3ee640b) * Fix ingest drone + err handling (#59) (8307750) * Fix code block lang (#62) (cffcbf0) * Get current branch from Snooty Data API plus allow override (#56) (6114e7c) * Clean up pages for embedding content (#60) (45ce277) * (DOCSP-31799): Parse definition lists (and other relevant content inside of `"target"` nodes) (#58) (2d7d1b3) * rename chat-ui package (#57) (dc7794f) * (DOCSP-31574): Create qualitative testing env (#55) (0a1c92a) * Add tests for data deletion (#48) (9b1417a) * Update Snooty Data Source to strip 'index' (#50) (d6b0b2f) * Add ability to override Snooty project baseUrl (#52) (fccc8ca) * Split up tests + add utils (#53) (0dedc6f) * (DOCSP-31436) [Q&A] Investigate & improve streaming rate (#54) (cc90cd0) * (DOCSP-31120) [Q&A] separate further reading reference links from LLM response (#43) (a604d83) * (DOCSP-30863) [Q&A] type check requests (#23) (5f908c0) * Streamline/nitpick/bikeshed the PR template (#51) (28b4e1b) * (DOCSP-31601): Update server base URL (#49) (1c60a0b) * (DOCSP-31573): Update prompts to address bugs (#47) (a366803) * (DOCSP-30602): Ingest service prod pipeline (#42) (8e60755) * Update pull_request_template.md (ab166a7) * Monorepo docker build fix (#46) (1dcef28) * (DOCSP-31284): Integrate prod Snooty Data API (#44) (f80ce51) * (DOCSP-31360) [UI] Conditional Streaming (#45) (34490bb) * update root package-lock (578b06b) * rename chat-ui package (d7b6dcf) * Add shared dev script to monorepo root (67811d1) * Streaming works (#25) (c173ba2) * (DOCSP-30601): Create production release system (#41) (7472c9d) * (DOCSP-31304): Implement 'all' command (#40) (f0c4cf2) * (DOCSP-31201) [UI] Updated Legalese (#39) (39c0f97) * (DOCSP-30584): ingest service CI testing and staging drone pipeline (#38) (d2e8325) * Fix + improve devcenter source tests (#37) (be0a5eb) * Fix URL building for devcenter paths (#36) (4c75175) * Add logic for ingest resumption (#35) (0f650c2) * stricter LLM to reduce halucination risk + change demo page title (#33) (cfb4628) * (DOCSP-30890) [UI] Make component mobile responsive (#29) (66080a7) * Add minor optimization for rate-limited embed requests (#34) (7547e31) * (DOCSP-31121): Add request tracking across logs (#32) (7fdc1e5) * (DOCSP-30988): Implement ingest E2E (#28) (8467989) * (DOCSP-30843): IP Address tracking and validation (#21) (7335672) * Update seed-data script (4f7a2f7) * (DOCSP-31050): Fix CI tests + run tests on PR (#30) (74fe090) * (DOCSP-30953 & DOCSP-30945): Error and edge case handling (#24) (af1b845) * Add PR template to repo (#26) (5231ef0) * Fix tests (#27) (58599ea) * more specific linter targets (18d2f5d) * Move and refactor EmbeddedContent/Store and clean up core services (#19) (43078a2) * separate format from lint (3a817b7) * remove console logs (#22) (20027ce) * Add lists to LGMarkdown (#20) (14cf986) * (DOCSP-30964): Staging build (#18) (1163ff6) * Fix renames (945337e) * Rename "chunks" to "embedded content" (#17) (a18bb8d) * Implement Data Sources (#12) (e9c5784) * Move reusable pieces to chat-core (#14) (3babf35) * (DOCSP-30577): POST `/conversations/:conversationId/messages/:messageId/rating` endpoint (#16) (c659173) * E2E (#15) (c85a2aa) * failed message handling (16acff7) * better prompt validation error handling (6408669) * UI tweaks (f803938) * (DOCSP-30921) [UI] Add prompt validation & error handling (1) (cb98ec7) * (DOCSP-30616): `POST /conversations/:conversationId/messages/` route, no streaming (#13) (8451c08) * Add publishing pipeline (42ab17e) * (DOCSP-30575): POST /conversations/ route (#11) (d609171) * (DOCSP-30619, DOCSP-30758, DOCSP-30722): Add MongoDB integration and conversation/content services (#9) (020a501) * Implement ingest store/database (#8) (5b9f8f6) * Add initial design docs (#1) (31d9069) * (DOCSP-30585): Generate search index script (#10) (458403f) * Merge branch 'demodalize' of github.com:nlarew/docs-chatbot (8d46f37) * better avatars (4c7c712) * tweak card heading (40d1dff) * loading ui (58256da) * (DOCSP-30617): OpenAI LLM service (#7) (d4af1f9) * refactor + loading states (4418f8d) * (DOCSP-30618): Create embeddings service (#6) (c54d123) * mock out Q&A API (9d80c20) * Refactor to use a single chat input (6ac351a) * Animation works except for Card ref (2bc2b41) * Rename & refactor (b5385af) * Send button for all inputs with text (9ca1873) * Move input state up (664b456) * Animation works (b8c2f42) * Use card modal WIP (27c2392) * cleanup (d96bcf0) * Fix bad imports (524f896) * (DOCSP-30576): Stub out Q&A server (#4) (899c0a3) * LGMarkdown (33596cf) * Working on LGMarkdown (897b141) * consistent models with backend (319b2ae) * UI lint pass (d8051d9) * Leafie avatar (ebd660b) * Don't require React Context for conversations (86c90b6) * fix modal style (6ae1500) * Add storybook (73a4ec3) * Modal (cac03d2) * fix banner width (088d858) * Merge pull request #3 from mongodb/DOCSP-30565 (5264da5) * Update .drone.yml (a9e3ba5) * working staging build + updated untested prod (fe16f6c) * remove uncreated env secrets (318c501) * fix broken registry path (a1e1dff) * change release (072100f) * update environment info (ff22f4f) * empty state (a66ce15) * fix value file location (ee4f06b) * fix pipeline (89f9e8b) * Trigger build (408d7dc) * Add (broken) tests (329d9e0) * Trigger build (6cae68f) * Fix context (7d213c0) * update path to docker file (f0edac3) * trigger build (e5e64c0) * more debugging (2dfb919) * Use dependency injection for PageStore/ChunkStore interfaces (89577b8) * Remove start comment (287182d) * move drone file (7453f99) * trigger build? (95579f6) * test deploy pipeline (aab4632) * add cta (db01cdb) * apply prettier (c49d368) * remove unused imports (85f6ab3) * add prettier (c1cf166) * combined props (ae9d66c) * chat state works (e29ee25) * add temp aria-labelledby prop (a0d4695) * Scaffold the ingest CLI (81a5c41) * Update ingest/README (65d2902) * chatbot-ui -> chat-ui (4340d1a) * Merge pull request #2 from mongodben/DOCSP-30486 (17662e6) * Bootstrap ingest CLI tool (0590da2) * Add chatbot-ui (997bec5) * set up QA server project structure (7cd25d5) * Delete specs directory (6b06d68) * ready repo for first push (2b250da)

* (DOCSP-32482): Build out qualitative tests (#201) * touch up qualitative test suite + add tags * add phase 1 general draft * add general tests * remove console.log * add rename files * sept 23 question yaml generator * update scripts to handle different testing scenarios * add round 2 test draft * update test types * update more test cases * add all qualitative tests * fix bad case * (DOCSP-33744): update QA server DB name for latest snapshot (#210) update QA server DB name * dev QoL scripts * (DOCSP-32603): Add Terraform data source (#205) * add terraform data source * update for more pageS * implement CB feedback * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) * update repo to use lerna versioning * update contributing guide * update release conf * add release-it to chat-core * update version * Release mongodb-rag-core v0.0.1 * Release mongodb-rag-core v0.0.2-2 * add changelog setup * release notes/changelog * Add changelog * Release mongodb-rag-core v0.0.2-3 * Add changelog (4a9af69) * release notes/changelog (c9b3b0f) * add changelog setup (3e90095) * all * inside * add outside commit (shouldn't see) * inside (should see) * update release conf * remove from / to from github * Release mongodb-rag-core v0.0.2-4 * remove from / to from github (881e474) * update release conf (0477bc2) * inside (should see) (7f19587) * inside (c22271b) * all (90b568a) * configure so that publishes package from creation of gh tag via drone * add .drone pipelines to deploy new npm packages * Release mongodb-rag-core v0.0.2-5 * configure so that publishes package from creation of gh tag via drone (f9df609) * update ingest conf * update deps * add non prerelease version * Release mongodb-rag-ingest v0.1.3 * update core + ingest release conf * separate chat server package and implementation + update drone file * remove release-it from monorepo conf * update drone names * add new npm creds * update pub flow to speed up testing * Release mongodb-rag-core v0.0.2-6 * update core + ingest release conf (5559403) * update package files to use * update core version in server * working chat server * remove chat-server tests from prod build * remove chatbot-server-mongodb-public tests from prod build * make ingest tests pass * fix broken server tests * fix flaky tests * fix build issues * Release mongodb-rag-core v0.0.2-7 * add publish env vars * Release mongodb-rag-core v0.0.2-8 * trigger ingest staging build * hack drone to trigger builds * trigger ingest build again * retry drone * fix build commands * simplify dockerfile * make ingest a node executable * Release mongodb-rag-core v0.0.2 * update package.json files with files to include in release * Release mongodb-rag-core v0.0.3 * update package.json files with files to include in release (e4567ed) * Add simple readme to core * Release mongodb-rag-core v0.0.4 * Add simple readme to core (729f44a) * update ingest version of core to latest working * update monorepo chatcore version * Release mongodb-rag-ingest v0.1.4 * update ingest version of core to latest working (5a7b346) * update package.json files with files to include in release (e4567ed) * make ingest a node executable (da4011f) * fix build commands (37d746b) * retry drone (bbbdd82) * trigger ingest build again (1315732) * trigger ingest staging build (33c7f34) * fix flaky tests (9dc7153) * make ingest tests pass (00c9065) * update core + ingest release conf (5559403) * update dependencies to latest in mongodb ingest * update package-lock.json w new deps * update package-lock * trigger build of ingest-mongodb-public to test drone * make CLI entry point executable * update lock * Release mongodb-rag-ingest v0.1.5 * make CLI entry point executable (00b57d3) * update mongodb ingest to use latest ingest CLI * update executable path * update chat-server.dockerfile * Trigger staging build of mongodb server * update chat-core dependency * fix build err from test * update chatbot ui out destination * update npm chat-core publish config * Release mongodb-rag-core v0.0.5-0 * Release mongodb-rag-core v0.0.5-1 * Release mongodb-rag-core v0.0.5-0 (580e8ff) * fix drone script * Release mongodb-rag-core v0.0.5-2 * update drone publication pipelines * Release mongodb-rag-ingest v0.1.6-0 * fix drone file * Release mongodb-rag-ingest v0.1.6-1 * bunlde chat-core in ingest * update lock file * Release mongodb-rag-ingest v0.1.6-2 * bunlde chat-core in ingest (3f763fe) * Add missing typechat dep * make ingest reexport core * remove unused bundled dep * Release mongodb-rag-core v0.0.5-3 * Add missing typechat dep (4bf2594) * Release mongodb-rag-core v0.0.5 * update dependent packages * update lock * update lock * Release mongodb-rag-ingest v0.1.6 * update dependent packages (50a43db) * remove unused bundled dep (87f6f7e) * make ingest reexport core (5b0fd8e) * update drone * remove test from release-it b/c covered in CI * remove unused dep * update release conf * update core version * update lock * Release mongodb-chatbot-server v0.1.9 * update lock (e45350d) * update core version (7d15d35) * update release conf (05a6686) * remove unused dep (12c73a3) * remove test from release-it b/c covered in CI (99f2a9b) * update drone (932c9a8) * Release mongodb-rag-ingest v0.1.6 (d9e3ea9) * update lock (bd718e6) * update lock (dac1773) * update dependent packages (50a43db) * Release mongodb-rag-core v0.0.5 (55195cb) * Release mongodb-rag-core v0.0.5-3 (b765423) * remove unused bundled dep (87f6f7e) * make ingest reexport core (5b0fd8e) * Add missing typechat dep (4bf2594) * Release mongodb-rag-ingest v0.1.6-2 (a22fbdf) * update lock file (def3cb7) * bunlde chat-core in ingest (3f763fe) * Release mongodb-rag-ingest v0.1.6-1 (fdd536c) * fix drone file (9b16f20) * Release mongodb-rag-ingest v0.1.6-0 (767efac) * update drone publication pipelines (3d761a6) * Release mongodb-rag-core v0.0.5-2 (a5689a2) * fix drone script (f711595) * Release mongodb-rag-core v0.0.5-1 (0a3745a) * Release mongodb-rag-core v0.0.5-0 (580e8ff) * update npm chat-core publish config (5be4922) * update chatbot ui out destination (d3e20a2) * fix build err from test (af0cce0) * update chat-core dependency (069fb9b) * Trigger staging build of mongodb server (a438a82) * update chat-server.dockerfile (81999da) * update executable path (5b06c12) * update mongodb ingest to use latest ingest CLI (b21441f) * Release mongodb-rag-ingest v0.1.5 (bf852d1) * update lock (70a8c65) * make CLI entry point executable (00b57d3) * trigger build of ingest-mongodb-public to test drone (96b4206) * update package-lock (ff83848) * update package-lock.json w new deps (f4d6579) * update dependencies to latest in mongodb ingest (0d6e48a) * Release mongodb-rag-ingest v0.1.4 (7b7e8de) * update monorepo chatcore version (80ebd61) * update ingest version of core to latest working (5a7b346) * Release mongodb-rag-core v0.0.4 (e04b754) * Add simple readme to core (729f44a) * Release mongodb-rag-core v0.0.3 (230ab97) * update package.json files with files to include in release (e4567ed) * Release mongodb-rag-core v0.0.2 (e5e6ac1) * make ingest a node executable (da4011f) * simplify dockerfile (e2d8751) * fix build commands (37d746b) * retry drone (bbbdd82) * trigger ingest build again (1315732) * hack drone to trigger builds (88e3255) * trigger ingest staging build (33c7f34) * Release mongodb-rag-core v0.0.2-8 (2258731) * add publish env vars (1ed87ad) * Release mongodb-rag-core v0.0.2-7 (c5914d3) * fix build issues (965a9e0) * fix flaky tests (9dc7153) * fix broken server tests (e4b04b9) * make ingest tests pass (00c9065) * remove chatbot-server-mongodb-public tests from prod build (d95c984) * remove chat-server tests from prod build (4c0cd22) * working chat server (0248a11) * update core version in server (d236f91) * update package files to use (7d3e5b8) * Release mongodb-rag-core v0.0.2-6 (40688a7) * update pub flow to speed up testing (8e0409c) * add new npm creds (cf6084a) * update drone names (47dd13c) * remove release-it from monorepo conf (ff4434c) * separate chat server package and implementation + update drone file (4441cbf) * update core + ingest release conf (5559403) * Release mongodb-rag-ingest v0.1.3 (9a7a2f9) * add non prerelease version (756d1a7) * update deps (7f4ac98) * update ingest conf (88c47ee) * Release mongodb-rag-core v0.0.2-5 (ba238e6) * add .drone pipelines to deploy new npm packages (f4a8210) * configure so that publishes package from creation of gh tag via drone (f9df609) * Release mongodb-rag-core v0.0.2-4 (d626d91) * remove from / to from github (881e474) * update release conf (0477bc2) * inside (should see) (7f19587) * add outside commit (shouldn't see) (feccfe7) * inside (c22271b) * all (90b568a) * Release mongodb-rag-core v0.0.2-3 (a90c1ba) * Add changelog (4a9af69) * release notes/changelog (c9b3b0f) * add changelog setup (3e90095) * Release mongodb-rag-core v0.0.2-2 (bcf3c9c) * Release mongodb-rag-core v0.0.1 (a3a1355) * update version (f6e1d8f) * add release-it to chat-core (629de3e) * update release conf (73308a6) * update contributing guide (11ee87b) * update repo to use lerna versioning (f7c7285) * Merge remote-tracking branch 'upstream/main' into oss-refactor-pt-2 (412da6a) * Ingest plugins (#211) (0206bee) * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) (b8541a7) * OSS Chat Server (#206) (820e249) * (DOCSP-32603): Add Terraform data source (#205) (0d64743) * dev QoL scripts (3813ad7) * (DOCSP-33744): update QA server DB name for latest snapshot (#210) (5df2bf0) * (DOCSP-32482): Build out qualitative tests (#201) (0535a89) * Organize some files (#207) (f8cdd3c) * Conversations refactor (#199) (5a60a15) * Release ingest-v0.1.2 (#204) (ed34507) * [Snyk] Upgrade urllib from 3.19.1 to 3.19.2 (#203) (743cc01) * Add Rust and Entity Framework Snooty data sources (#202) (ff15139) * [Snyk] Upgrade urllib from 3.19.0 to 3.19.1 (#200) (dea05e0) * Release chat-server v0.1.3 (ac542c8) * Use GITHUB_TOKEN for chat-ui releases (38a9e1a) * [Snyk] Upgrade urllib from 3.18.1 to 3.19.0 (#197) (dd2baaf) * Do not fail query on rephrase fail (#196) (8b7c60a) * Release chat-ui v0.0.8 (03bf7cb) * Bundle new @lg-chat components (2f1f11b) * Release chat-ui v0.0.7 (d082a63) * Unify release-it config (419422e) * Release chat-server v0.1.2 (b0b425c) * [Snyk] Upgrade urllib from 3.18.0 to 3.18.1 (#194) (2c98ca4) * [DOCSP-32950] Create a DevCenter Chatbot Component (#193) (c219d03) * (DOCSP-33445) [ANALYTICS] Scrubbed Views (#195) (8ef2bd7) * (DOCSP-33419) [ANALYTICS] Operational Views (#192) (aae8259) * Release scripts v0.1.2 (fbf2bbf) * Add whether message was deemed "Do not answer" to conversation (#191) (955b722) * (DOCSP-31104) [UI] Add tests to FE services (#190) (d3e2000) * Release ingest v0.1.1 (5b06cd2) * Update package-lock.json (a568d00) * Replace GitHub data source (#188) (747d08e) * Update env vars for analyze script (0a46085) * Add scrubbed message analyzer (#187) (9cef6b8) * (DOCSP-33291) [UI] Add tck tracking to links (#185) (acfb6a1) * Update README.md (a38cbf7) * Release 0.1.1 (8ee27c1) * Fix typo in scripts build (0534458) * Release 0.1.0 (09fa262) * Add release-it to scripts (9ea2e71) * Trigger drone run for scripts service (b1ffcc8) * Fix drone scripts (#186) (9bae8e9) * Cron scrub (#184) (d67dda9) * (DOCSP-33273) Open Source Licensing (#183) (00611bf) * De-flake SnootyProjectsInfo test (#181) (9f7250e) * (DOCSP-32930): Add pymongo metadata (#179) (6694209) * Update release-it git add config to add monorepo stuff (679e447) * Update package-lock in monorepo for release (e7308ef) * Release 0.1.1 (d60d14b) * Update release-it pre-hooks (718b96d) * Run npm audit fix (#178) (c74e1a9) * Remove `seed-content` package (#157) (1b9ac43) * Add scrub message script (#177) (ee2f864) * Remove extra padding around initial input (9dc27ad) * (DOCSP-33094): Save conversation embeddings (#175) (9c6e79d) * Ingest release 0.1.0 (#174) (4928eed) * Release chat-ui-v0.0.6 (2bf2a0e) * (DOCSP-33059) [UI] Fix suggested prompts & other LG Chat tweaks (#171) (cde1b74) * chat server v0.1.0 (#170) (ba69ad8) * new commit for build (56ed01e) * trigger ingest build (4052af9) * clean up spec + remove ip address validation from rate msg (#166) (28cf222) * UI Tweaks (#159) (53b392f) * Delete chat-server/trigger.me (#165) (5542f41) * update repo name in code (#163) (f9628bb) * Update README.md (5ad9cd0) * Launch pre releases (#162) (9b2380b) * Expand Contributing Guide and `ingest` README (#161) (30434e3) * filter out very small chunks (#160) (d952c6d) * (DOCSP-31928): Implement numbered lists (#154) (f41258d) * clarify ingest readme (115b00b) * move git install to correct docker stage (40600cb) * (DOCSP-32991, DOCSP-32992): `GitDataSource`, functionality for converting HTML to Markdown, ingest Scala, Java Reactive Streams, and C docs (#152) (7118444) * (DOCSP-32915) [UI] Streamer bugs out when switching tabs (#156) (91c288d) * (DOCSP-33090) [UI] Remove experimental badge from input on focus (#155) (a3ea236) * (DOCSP-32993, DOCSP-32906, DOCSP-32993, DOCSP-32990): Data sources for MD on Github + data ingestion of related content (#148) (7daed63) * change recommended prompt (#150) (2521f08) * add missing env example (#153) (1201a34) * [Snyk] Upgrade mongodb from 5.8.0 to 5.8.1 (#151) (966a4bf) * make preprocessor less biased toward shell (#149) (886cbe5) * (DOCSP-32616): Refactor config, add library, and log config (#143) (4fcfc58) * add default + remove unused style (0296b42) * Update chat-ui release config (3174b61) * Release chat-ui-v0.0.5 (65727f7) * Add root release:chat-ui command (4621989) * (DOCSP-32757) [UI] Update to Responsive LG Chat Components (#145) (81b70bb) * (DOCSP-32670): Basic server load testing (#136) (282c947) * [Snyk] Upgrade urllib from 3.17.2 to 3.18.0 (#147) (aefa9e0) * (DOCSP-32756): Implement headings in tables (#140) (ea416ce) * custom/synthetic/meta data source content (#127) (d687312) * [Snyk] Upgrade urllib from 3.17.1 to 3.17.2 (#144) (209135a) * (DOCSP-32673) [UI] Suppress focus to prevent modal reopen (#139) (43b8a98) * (DOCSP-32936): Increase single IP rate limits to be more line w firewall rate limits (#141) (87d8785) * (DOCSP-31863): Fix flaky test, refactor core llm service + server implementation (#142) (584a5e3) * (DOCSP-32916): Conversation service rate limiting + slow down (#138) (a88a5d5) * (DOCSP-32914): Stop unindenting context (#137) (27630c6) * (DOCSP-31285): Ingest PyMongo docs (#118) (e04c10b) * (DOCSP-32694) Dark mode: use LG palette & provider + cleanup (a95f5c5) * (DOCSP-32694) [UI] Add darkMode prop (#129) (820229c) * (DOCSP-32896) Remove mention of <CONTEXT> to reduce damage of prompt leakage (#135) (1484426) * Update internal docs (#134) (a3b2f56) * Handle Snooty Data API tombstones (#133) (a775da4) * (DOCSP-32867): Update QA DB name for sept 1 release (#131) (ce4fa3f) * (DOCSP-32868): only boost 2 results + order results by score (#130) (5230992) * (DOCSP-32856) Add a toggle for streamed vs awaited responses to the demo (#126) (94e8a20) * (DOCSP-32808): k8s production readiness (#125) (2cdf539) * Upgrade azure openai (#128) (6b85e63) * (DOCSP-32759) [UI] Use text instead of a banner for "validate answers" (#124) (302b3c7) * (DOCSP-32604) More helpful max messages error message (#123) (1be4087) * (DOCSP-32254): Include information about latest product versions + minor prompt engineering (#116) (2e40123) * (DOCSP-32657): Remove same IP address validation on conversations (#112) (731ec20) * (DOCSP-31108): Add prod index scripts (#120) (3efe895) * (DOCSP-32807): Add pm2 for running server (#122) (f659e4b) * [Snyk] Security upgrade mongodb from 5.7.0 to 5.8.0 (#121) (c985fef) * Autoscaling fix (#119) (641199a) * Source GIT_COMMIT env variable (#95) (647b00a) * Allow DOP origins in staging (#117) (38a9b66) * (DOCSP-32074): Handle dev center page metadata (#96) (e283121) * Release chat-ui-v0.0.4 (d3bad60) * (DOCSP-32371): Allow for embedded content re-ingest after changes to chunking algorithm (#114) (8b9d68e) * (DOCSP-32279): Update prompts to not mention context when don't know answer (#113) (71ca44b) * (DOCSP-32688) Stop dereferencing spec (#115) (add0dd8) * (DOCSP-31438): Horizontal pod autoscaling (#110) (3c124d4) * fix registry (d52ff7d) * Release chat-ui-v0.0.3 (f8113b7) * Add chat-ui release-it config (731e3d6) * Switch Chatbot to default export (b5db7f0) * Chat UI v0.0.2 (#111) (7e5d684) * (DOCSP-32659): Make server call its own base url if not given a full URL path (#109) (6cdd026) * Allow users to override server URL via props (#108) (708e614) * (DOCSP-32547): Update prod env to use dedicated Azure deployment (#104) (ac93735) * Prod drone fixes (#107) (4df6432) * Update QA server DB (#105) (a12e7c9) * (DOCSP-32452): Dev Center data source remove `<img>` and `<div>` tags + youtube directive (#93) (c5dac2f) * (DOCSP-32596): Set up CORS on the server (#103) (76c9530) * (DOCSP-32104): Do not serve demo site in prod environment (#101) (5f0ea11) * (DOCSP-32247): Fix preprocessed content not added to DB (#100) (57cc24e) * fix handling of openapi specs (#99) (6f9b0e4) * fix hanging promise in async recursive operation (#98) (c974058) * Drone fix (#97) (e54a272) * (DOCSP-32363) [UI] handle cases where the LLM stops mid code example (#85) (9e26c7f) * (DOCSP-32155) [UI] UX Feedback (#81) (c34b246) * (DOCSP-32243) Handle and test pages with page-level code block select (#94) (f6c3f7e) * (DOCSP-32490) [UI] Link to current commit on GitHub (#90) (64dbd0d) * (DOCSP-32242): Handle tabbed Snooty content (#92) (f87f67d) * Restore 'tags' and use 'tagsIn' only internally (#91) (035f018) * (DOCSP-32227): Improve OpenAPI spec ingestion (#88) (f369218) * (DOCSP-31343): Add system diagram and info for ingest (#86) (6e4402b) * (DOCSP-32362): Improve conversation request logging (#89) (b251e0d) * (DOCSP-32253): Add more semantically relevant product names to metadata (#83) (5869f22) * (DOCSP-32434) [UI] Add error text for non-modal input (#87) (5de9282) * (DOCSP-32206): Strip comments from rst -> md (#84) (611a50d) * (DOCSP-32194): Fix drone and k8s for ingest/chat-server staging (#80) (b61c9bc) * Define engines to remove EBADENGINE warning (#82) (f333ef5) * (DOCSP-31440, DOCSP-32075): Add tags to chunk metadata (#78) (254ce2b) * (DOCSP-32217): Change chunker to chunk based on number of tokens + add max context tokens per message (#79) (e7c1cb6) * (DOCSP-32183 & DOCSP-32226): Automate test creation based on YAML file (#77) (30eb660) * Add database index creation scripts (#75) (53f296f) * (DOCSP-31575): LLM qualitative testing framework (#74) (f222e54) * (DOCSP-31111) [UI] publish react component to npm (#73) (3baf40e) * Refactor `ConversationService` class as `makeConversationService` func (#76) (6cb9a83) * (DOCSP-32070): LLM preprocessing on user queries (#72) (6ff4219) * (DOCSP-31925): Support tables in parsed snooty MD (#67) (539d403) * Reduce ChatGPT API flakiness + faster timeout when flaky (#69) (8938599) * Update Chat UI Packaging (#71) (2e5a0fb) * (DOCSP-32066): Q&A server reduce amount of backoff on embedding (#68) (fe938b0) * (DOCSP-31639): fix timeout err in tests (#70) (741ea29) * Implement page titles and prepending frontmatter to chunks (#64) (cf56a26) * Fix JS error surfaced in the UI (#66) (5e6866a) * bundle LG Chat with component lib (c4fc213) * Modify ui config for pre-release (135d211) * (DOCSP-31154) [UI] Handle timeout & "conversation too long" errors (#65) (1943608) * (DOCSP-31791): Boost manual in the results for short queries + make app config file (#63) (6449993) * (DOCSP-31523) [UI] Replace custom components with LG Chat (#61) (3ee640b) * Fix ingest drone + err handling (#59) (8307750) * Fix code block lang (#62) (cffcbf0) * Get current branch from Snooty Data API plus allow override (#56) (6114e7c) * Clean up pages for embedding content (#60) (45ce277) * (DOCSP-31799): Parse definition lists (and other relevant content inside of `"target"` nodes) (#58) (2d7d1b3) * rename chat-ui package (#57) (dc7794f) * (DOCSP-31574): Create qualitative testing env (#55) (0a1c92a) * Add tests for data deletion (#48) (9b1417a) * Update Snooty Data Source to strip 'index' (#50) (d6b0b2f) * Add ability to override Snooty project baseUrl (#52) (fccc8ca) * Split up tests + add utils (#53) (0dedc6f) * (DOCSP-31436) [Q&A] Investigate & improve streaming rate (#54) (cc90cd0) * (DOCSP-31120) [Q&A] separate further reading reference links from LLM response (#43) (a604d83) * (DOCSP-30863) [Q&A] type check requests (#23) (5f908c0) * Streamline/nitpick/bikeshed the PR template (#51) (28b4e1b) * (DOCSP-31601): Update server base URL (#49) (1c60a0b) * (DOCSP-31573): Update prompts to address bugs (#47) (a366803) * (DOCSP-30602): Ingest service prod pipeline (#42) (8e60755) * Update pull_request_template.md (ab166a7) * Monorepo docker build fix (#46) (1dcef28) * (DOCSP-31284): Integrate prod Snooty Data API (#44) (f80ce51) * (DOCSP-31360) [UI] Conditional Streaming (#45) (34490bb) * update root package-lock (578b06b) * rename chat-ui package (d7b6dcf) * Add shared dev script to monorepo root (67811d1) * Streaming works (#25) (c173ba2) * (DOCSP-30601): Create production release system (#41) (7472c9d) * (DOCSP-31304): Implement 'all' command (#40) (f0c4cf2) * (DOCSP-31201) [UI] Updated Legalese (#39) (39c0f97) * (DOCSP-30584): ingest service CI testing and staging drone pipeline (#38) (d2e8325) * Fix + improve devcenter source tests (#37) (be0a5eb) * Fix URL building for devcenter paths (#36) (4c75175) * Add logic for ingest resumption (#35) (0f650c2) * stricter LLM to reduce halucination risk + change demo page title (#33) (cfb4628) * (DOCSP-30890) [UI] Make component mobile responsive (#29) (66080a7) * Add minor optimization for rate-limited embed requests (#34) (7547e31) * (DOCSP-31121): Add request tracking across logs (#32) (7fdc1e5) * (DOCSP-30988): Implement ingest E2E (#28) (8467989) * (DOCSP-30843): IP Address tracking and validation (#21) (7335672) * Update seed-data script (4f7a2f7) * (DOCSP-31050): Fix CI tests + run tests on PR (#30) (74fe090) * (DOCSP-30953 & DOCSP-30945): Error and edge case handling (#24) (af1b845) * Add PR template to repo (#26) (5231ef0) * Fix tests (#27) (58599ea) * more specific linter targets (18d2f5d) * Move and refactor EmbeddedContent/Store and clean up core services (#19) (43078a2) * separate format from lint (3a817b7) * remove console logs (#22) (20027ce) * Add lists to LGMarkdown (#20) (14cf986) * (DOCSP-30964): Staging build (#18) (1163ff6) * Fix renames (945337e) * Rename "chunks" to "embedded content" (#17) (a18bb8d) * Implement Data Sources (#12) (e9c5784) * Move reusable pieces to chat-core (#14) (3babf35) * (DOCSP-30577): POST `/conversations/:conversationId/messages/:messageId/rating` endpoint (#16) (c659173) * E2E (#15) (c85a2aa) * failed message handling (16acff7) * better prompt validation error handling (6408669) * UI tweaks (f803938) * (DOCSP-30921) [UI] Add prompt validation & error handling (1) (cb98ec7) * (DOCSP-30616): `POST /conversations/:conversationId/messages/` route, no streaming (#13) (8451c08) * Add publishing pipeline (42ab17e) * (DOCSP-30575): POST /conversations/ route (#11) (d609171) * (DOCSP-30619, DOCSP-30758, DOCSP-30722): Add MongoDB integration and conversation/content services (#9) (020a501) * Implement ingest store/database (#8) (5b9f8f6) * Add initial design docs (#1) (31d9069) * (DOCSP-30585): Generate search index script (#10) (458403f) * Merge branch 'demodalize' of github.com:nlarew/docs-chatbot (8d46f37) * better avatars (4c7c712) * tweak card heading (40d1dff) * loading ui (58256da) * (DOCSP-30617): OpenAI LLM service (#7) (d4af1f9) * refactor + loading states (4418f8d) * (DOCSP-30618): Create embeddings service (#6) (c54d123) * mock out Q&A API (9d80c20) * Refactor to use a single chat input (6ac351a) * Animation works except for Card ref (2bc2b41) * Rename & refactor (b5385af) * Send button for all inputs with text (9ca1873) * Move input state up (664b456) * Animation works (b8c2f42) * Use card modal WIP (27c2392) * cleanup (d96bcf0) * Fix bad imports (524f896) * (DOCSP-30576): Stub out Q&A server (#4) (899c0a3) * LGMarkdown (33596cf) * Working on LGMarkdown (897b141) * consistent models with backend (319b2ae) * UI lint pass (d8051d9) * Leafie avatar (ebd660b) * Don't require React Context for conversations (86c90b6) * fix modal style (6ae1500) * Add storybook (73a4ec3) * Modal (cac03d2) * fix banner width (088d858) * Merge pull request #3 from mongodb/DOCSP-30565 (5264da5) * Update .drone.yml (a9e3ba5) * working staging build + updated untested prod (fe16f6c) * remove uncreated env secrets (318c501) * fix broken registry path (a1e1dff) * change release (072100f) * update environment info (ff22f4f) * empty state (a66ce15) * fix value file location (ee4f06b) * fix pipeline (89f9e8b) * Trigger build (408d7dc) * Add (broken) tests (329d9e0) * Trigger build (6cae68f) * Fix context (7d213c0) * update path to docker file (f0edac3) * trigger build (e5e64c0) * more debugging (2dfb919) * Use dependency injection for PageStore/ChunkStore interfaces (89577b8) * Remove start comment (287182d) * move drone file (7453f99) * trigger build? (95579f6) * test deploy pipeline (aab4632) * add cta (db01cdb) * apply prettier (c49d368) * remove unused imports (85f6ab3) * add prettier (c1cf166) * combined props (ae9d66c) * chat state works (e29ee25) * add temp aria-labelledby prop (a0d4695) * Scaffold the ingest CLI (81a5c41) * Update ingest/README (65d2902) * chatbot-ui -> chat-ui (4340d1a) * Merge pull request #2 from mongodben/DOCSP-30486 (17662e6) * Bootstrap ingest CLI tool (0590da2) * Add chatbot-ui (997bec5) * set up QA server project structure (7cd25d5) * Delete specs directory (6b06d68) * ready repo for first push (2b250da) * refactor chatbot server to use latest versions * remove trigger files * clean drone up after changes for development * update contributor guide * Update CONTRIBUTING.md Co-authored-by: Chris Bush <chris.bush@mongodb.com> * Update chat-core/src/extractFrontMatter.ts --------- Co-authored-by: Nick Larew <nick.larew@mongodb.com> Co-authored-by: Chris Bush <chris.bush@mongodb.com>

* Organize some files (#207) * Organize some files * Move test_data out of source * Undo over-barreling * Create test helper directory * OSS Chat Server (#206) * add OSS_TODOs * Add comments * more comment edits * fix build errs * remove abstraction from chat-core * update mongodb imports * Fix broken tests * start OpenAiEmbedFunc refactor * split data stores * update ingest w new stores * configurable reference links * Make max number messages & tokens in single RAG in conversation configurable * update tests w new route conf * update SomeStreamEventUsage * Refactor project as lib * wrap close statements so second doesn't fail even if first does * implement CB feedback * add gitignore * Ingest plugins (#211) * Ingest plugins WIP Externalize config WIP EmbedFunc -> Embedder Continue dev Fix test Update dockerfile and cron jobs Documentation Partialize configs * Fix test * WIP * Cleanup * Add testing * Further work to make it work? * Tidy * Update ingest/src/Config.ts Co-authored-by: Ben Perlmutter <90647379+mongodben@users.noreply.github.com> * Minor fix * Fix build? * Update Config usage * Address comments --------- Co-authored-by: Ben Perlmutter <90647379+mongodben@users.noreply.github.com> * OSS Refactor: Set up monorepo for package publication (#232) * (DOCSP-32482): Build out qualitative tests (#201) * touch up qualitative test suite + add tags * add phase 1 general draft * add general tests * remove console.log * add rename files * sept 23 question yaml generator * update scripts to handle different testing scenarios * add round 2 test draft * update test types * update more test cases * add all qualitative tests * fix bad case * (DOCSP-33744): update QA server DB name for latest snapshot (#210) update QA server DB name * dev QoL scripts * (DOCSP-32603): Add Terraform data source (#205) * add terraform data source * update for more pageS * implement CB feedback * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) * update repo to use lerna versioning * update contributing guide * update release conf * add release-it to chat-core * update version * Release mongodb-rag-core v0.0.1 * Release mongodb-rag-core v0.0.2-2 * add changelog setup * release notes/changelog * Add changelog * Release mongodb-rag-core v0.0.2-3 * Add changelog (4a9af69) * release notes/changelog (c9b3b0f) * add changelog setup (3e90095) * all * inside * add outside commit (shouldn't see) * inside (should see) * update release conf * remove from / to from github * Release mongodb-rag-core v0.0.2-4 * remove from / to from github (881e474) * update release conf (0477bc2) * inside (should see) (7f19587) * inside (c22271b) * all (90b568a) * configure so that publishes package from creation of gh tag via drone * add .drone pipelines to deploy new npm packages * Release mongodb-rag-core v0.0.2-5 * configure so that publishes package from creation of gh tag via drone (f9df609) * update ingest conf * update deps * add non prerelease version * Release mongodb-rag-ingest v0.1.3 * update core + ingest release conf * separate chat server package and implementation + update drone file * remove release-it from monorepo conf * update drone names * add new npm creds * update pub flow to speed up testing * Release mongodb-rag-core v0.0.2-6 * update core + ingest release conf (5559403) * update package files to use * update core version in server * working chat server * remove chat-server tests from prod build * remove chatbot-server-mongodb-public tests from prod build * make ingest tests pass * fix broken server tests * fix flaky tests * fix build issues * Release mongodb-rag-core v0.0.2-7 * add publish env vars * Release mongodb-rag-core v0.0.2-8 * trigger ingest staging build * hack drone to trigger builds * trigger ingest build again * retry drone * fix build commands * simplify dockerfile * make ingest a node executable * Release mongodb-rag-core v0.0.2 * update package.json files with files to include in release * Release mongodb-rag-core v0.0.3 * update package.json files with files to include in release (e4567ed) * Add simple readme to core * Release mongodb-rag-core v0.0.4 * Add simple readme to core (729f44a) * update ingest version of core to latest working * update monorepo chatcore version * Release mongodb-rag-ingest v0.1.4 * update ingest version of core to latest working (5a7b346) * update package.json files with files to include in release (e4567ed) * make ingest a node executable (da4011f) * fix build commands (37d746b) * retry drone (bbbdd82) * trigger ingest build again (1315732) * trigger ingest staging build (33c7f34) * fix flaky tests (9dc7153) * make ingest tests pass (00c9065) * update core + ingest release conf (5559403) * update dependencies to latest in mongodb ingest * update package-lock.json w new deps * update package-lock * trigger build of ingest-mongodb-public to test drone * make CLI entry point executable * update lock * Release mongodb-rag-ingest v0.1.5 * make CLI entry point executable (00b57d3) * update mongodb ingest to use latest ingest CLI * update executable path * update chat-server.dockerfile * Trigger staging build of mongodb server * update chat-core dependency * fix build err from test * update chatbot ui out destination * update npm chat-core publish config * Release mongodb-rag-core v0.0.5-0 * Release mongodb-rag-core v0.0.5-1 * Release mongodb-rag-core v0.0.5-0 (580e8ff) * fix drone script * Release mongodb-rag-core v0.0.5-2 * update drone publication pipelines * Release mongodb-rag-ingest v0.1.6-0 * fix drone file * Release mongodb-rag-ingest v0.1.6-1 * bunlde chat-core in ingest * update lock file * Release mongodb-rag-ingest v0.1.6-2 * bunlde chat-core in ingest (3f763fe) * Add missing typechat dep * make ingest reexport core * remove unused bundled dep * Release mongodb-rag-core v0.0.5-3 * Add missing typechat dep (4bf2594) * Release mongodb-rag-core v0.0.5 * update dependent packages * update lock * update lock * Release mongodb-rag-ingest v0.1.6 * update dependent packages (50a43db) * remove unused bundled dep (87f6f7e) * make ingest reexport core (5b0fd8e) * update drone * remove test from release-it b/c covered in CI * remove unused dep * update release conf * update core version * update lock * Release mongodb-chatbot-server v0.1.9 * update lock (e45350d) * update core version (7d15d35) * update release conf (05a6686) * remove unused dep (12c73a3) * remove test from release-it b/c covered in CI (99f2a9b) * update drone (932c9a8) * Release mongodb-rag-ingest v0.1.6 (d9e3ea9) * update lock (bd718e6) * update lock (dac1773) * update dependent packages (50a43db) * Release mongodb-rag-core v0.0.5 (55195cb) * Release mongodb-rag-core v0.0.5-3 (b765423) * remove unused bundled dep (87f6f7e) * make ingest reexport core (5b0fd8e) * Add missing typechat dep (4bf2594) * Release mongodb-rag-ingest v0.1.6-2 (a22fbdf) * update lock file (def3cb7) * bunlde chat-core in ingest (3f763fe) * Release mongodb-rag-ingest v0.1.6-1 (fdd536c) * fix drone file (9b16f20) * Release mongodb-rag-ingest v0.1.6-0 (767efac) * update drone publication pipelines (3d761a6) * Release mongodb-rag-core v0.0.5-2 (a5689a2) * fix drone script (f711595) * Release mongodb-rag-core v0.0.5-1 (0a3745a) * Release mongodb-rag-core v0.0.5-0 (580e8ff) * update npm chat-core publish config (5be4922) * update chatbot ui out destination (d3e20a2) * fix build err from test (af0cce0) * update chat-core dependency (069fb9b) * Trigger staging build of mongodb server (a438a82) * update chat-server.dockerfile (81999da) * update executable path (5b06c12) * update mongodb ingest to use latest ingest CLI (b21441f) * Release mongodb-rag-ingest v0.1.5 (bf852d1) * update lock (70a8c65) * make CLI entry point executable (00b57d3) * trigger build of ingest-mongodb-public to test drone (96b4206) * update package-lock (ff83848) * update package-lock.json w new deps (f4d6579) * update dependencies to latest in mongodb ingest (0d6e48a) * Release mongodb-rag-ingest v0.1.4 (7b7e8de) * update monorepo chatcore version (80ebd61) * update ingest version of core to latest working (5a7b346) * Release mongodb-rag-core v0.0.4 (e04b754) * Add simple readme to core (729f44a) * Release mongodb-rag-core v0.0.3 (230ab97) * update package.json files with files to include in release (e4567ed) * Release mongodb-rag-core v0.0.2 (e5e6ac1) * make ingest a node executable (da4011f) * simplify dockerfile (e2d8751) * fix build commands (37d746b) * retry drone (bbbdd82) * trigger ingest build again (1315732) * hack drone to trigger builds (88e3255) * trigger ingest staging build (33c7f34) * Release mongodb-rag-core v0.0.2-8 (2258731) * add publish env vars (1ed87ad) * Release mongodb-rag-core v0.0.2-7 (c5914d3) * fix build issues (965a9e0) * fix flaky tests (9dc7153) * fix broken server tests (e4b04b9) * make ingest tests pass (00c9065) * remove chatbot-server-mongodb-public tests from prod build (d95c984) * remove chat-server tests from prod build (4c0cd22) * working chat server (0248a11) * update core version in server (d236f91) * update package files to use (7d3e5b8) * Release mongodb-rag-core v0.0.2-6 (40688a7) * update pub flow to speed up testing (8e0409c) * add new npm creds (cf6084a) * update drone names (47dd13c) * remove release-it from monorepo conf (ff4434c) * separate chat server package and implementation + update drone file (4441cbf) * update core + ingest release conf (5559403) * Release mongodb-rag-ingest v0.1.3 (9a7a2f9) * add non prerelease version (756d1a7) * update deps (7f4ac98) * update ingest conf (88c47ee) * Release mongodb-rag-core v0.0.2-5 (ba238e6) * add .drone pipelines to deploy new npm packages (f4a8210) * configure so that publishes package from creation of gh tag via drone (f9df609) * Release mongodb-rag-core v0.0.2-4 (d626d91) * remove from / to from github (881e474) * update release conf (0477bc2) * inside (should see) (7f19587) * add outside commit (shouldn't see) (feccfe7) * inside (c22271b) * all (90b568a) * Release mongodb-rag-core v0.0.2-3 (a90c1ba) * Add changelog (4a9af69) * release notes/changelog (c9b3b0f) * add changelog setup (3e90095) * Release mongodb-rag-core v0.0.2-2 (bcf3c9c) * Release mongodb-rag-core v0.0.1 (a3a1355) * update version (f6e1d8f) * add release-it to chat-core (629de3e) * update release conf (73308a6) * update contributing guide (11ee87b) * update repo to use lerna versioning (f7c7285) * Merge remote-tracking branch 'upstream/main' into oss-refactor-pt-2 (412da6a) * Ingest plugins (#211) (0206bee) * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) (b8541a7) * OSS Chat Server (#206) (820e249) * (DOCSP-32603): Add Terraform data source (#205) (0d64743) * dev QoL scripts (3813ad7) * (DOCSP-33744): update QA server DB name for latest snapshot (#210) (5df2bf0) * (DOCSP-32482): Build out qualitative tests (#201) (0535a89) * Organize some files (#207) (f8cdd3c) * Conversations refactor (#199) (5a60a15) * Release ingest-v0.1.2 (#204) (ed34507) * [Snyk] Upgrade urllib from 3.19.1 to 3.19.2 (#203) (743cc01) * Add Rust and Entity Framework Snooty data sources (#202) (ff15139) * [Snyk] Upgrade urllib from 3.19.0 to 3.19.1 (#200) (dea05e0) * Release chat-server v0.1.3 (ac542c8) * Use GITHUB_TOKEN for chat-ui releases (38a9e1a) * [Snyk] Upgrade urllib from 3.18.1 to 3.19.0 (#197) (dd2baaf) * Do not fail query on rephrase fail (#196) (8b7c60a) * Release chat-ui v0.0.8 (03bf7cb) * Bundle new @lg-chat components (2f1f11b) * Release chat-ui v0.0.7 (d082a63) * Unify release-it config (419422e) * Release chat-server v0.1.2 (b0b425c) * [Snyk] Upgrade urllib from 3.18.0 to 3.18.1 (#194) (2c98ca4) * [DOCSP-32950] Create a DevCenter Chatbot Component (#193) (c219d03) * (DOCSP-33445) [ANALYTICS] Scrubbed Views (#195) (8ef2bd7) * (DOCSP-33419) [ANALYTICS] Operational Views (#192) (aae8259) * Release scripts v0.1.2 (fbf2bbf) * Add whether message was deemed "Do not answer" to conversation (#191) (955b722) * (DOCSP-31104) [UI] Add tests to FE services (#190) (d3e2000) * Release ingest v0.1.1 (5b06cd2) * Update package-lock.json (a568d00) * Replace GitHub data source (#188) (747d08e) * Update env vars for analyze script (0a46085) * Add scrubbed message analyzer (#187) (9cef6b8) * (DOCSP-33291) [UI] Add tck tracking to links (#185) (acfb6a1) * Update README.md (a38cbf7) * Release 0.1.1 (8ee27c1) * Fix typo in scripts build (0534458) * Release 0.1.0 (09fa262) * Add release-it to scripts (9ea2e71) * Trigger drone run for scripts service (b1ffcc8) * Fix drone scripts (#186) (9bae8e9) * Cron scrub (#184) (d67dda9) * (DOCSP-33273) Open Source Licensing (#183) (00611bf) * De-flake SnootyProjectsInfo test (#181) (9f7250e) * (DOCSP-32930): Add pymongo metadata (#179) (6694209) * Update release-it git add config to add monorepo stuff (679e447) * Update package-lock in monorepo for release (e7308ef) * Release 0.1.1 (d60d14b) * Update release-it pre-hooks (718b96d) * Run npm audit fix (#178) (c74e1a9) * Remove `seed-content` package (#157) (1b9ac43) * Add scrub message script (#177) (ee2f864) * Remove extra padding around initial input (9dc27ad) * (DOCSP-33094): Save conversation embeddings (#175) (9c6e79d) * Ingest release 0.1.0 (#174) (4928eed) * Release chat-ui-v0.0.6 (2bf2a0e) * (DOCSP-33059) [UI] Fix suggested prompts & other LG Chat tweaks (#171) (cde1b74) * chat server v0.1.0 (#170) (ba69ad8) * new commit for build (56ed01e) * trigger ingest build (4052af9) * clean up spec + remove ip address validation from rate msg (#166) (28cf222) * UI Tweaks (#159) (53b392f) * Delete chat-server/trigger.me (#165) (5542f41) * update repo name in code (#163) (f9628bb) * Update README.md (5ad9cd0) * Launch pre releases (#162) (9b2380b) * Expand Contributing Guide and `ingest` README (#161) (30434e3) * filter out very small chunks (#160) (d952c6d) * (DOCSP-31928): Implement numbered lists (#154) (f41258d) * clarify ingest readme (115b00b) * move git install to correct docker stage (40600cb) * (DOCSP-32991, DOCSP-32992): `GitDataSource`, functionality for converting HTML to Markdown, ingest Scala, Java Reactive Streams, and C docs (#152) (7118444) * (DOCSP-32915) [UI] Streamer bugs out when switching tabs (#156) (91c288d) * (DOCSP-33090) [UI] Remove experimental badge from input on focus (#155) (a3ea236) * (DOCSP-32993, DOCSP-32906, DOCSP-32993, DOCSP-32990): Data sources for MD on Github + data ingestion of related content (#148) (7daed63) * change recommended prompt (#150) (2521f08) * add missing env example (#153) (1201a34) * [Snyk] Upgrade mongodb from 5.8.0 to 5.8.1 (#151) (966a4bf) * make preprocessor less biased toward shell (#149) (886cbe5) * (DOCSP-32616): Refactor config, add library, and log config (#143) (4fcfc58) * add default + remove unused style (0296b42) * Update chat-ui release config (3174b61) * Release chat-ui-v0.0.5 (65727f7) * Add root release:chat-ui command (4621989) * (DOCSP-32757) [UI] Update to Responsive LG Chat Components (#145) (81b70bb) * (DOCSP-32670): Basic server load testing (#136) (282c947) * [Snyk] Upgrade urllib from 3.17.2 to 3.18.0 (#147) (aefa9e0) * (DOCSP-32756): Implement headings in tables (#140) (ea416ce) * custom/synthetic/meta data source content (#127) (d687312) * [Snyk] Upgrade urllib from 3.17.1 to 3.17.2 (#144) (209135a) * (DOCSP-32673) [UI] Suppress focus to prevent modal reopen (#139) (43b8a98) * (DOCSP-32936): Increase single IP rate limits to be more line w firewall rate limits (#141) (87d8785) * (DOCSP-31863): Fix flaky test, refactor core llm service + server implementation (#142) (584a5e3) * (DOCSP-32916): Conversation service rate limiting + slow down (#138) (a88a5d5) * (DOCSP-32914): Stop unindenting context (#137) (27630c6) * (DOCSP-31285): Ingest PyMongo docs (#118) (e04c10b) * (DOCSP-32694) Dark mode: use LG palette & provider + cleanup (a95f5c5) * (DOCSP-32694) [UI] Add darkMode prop (#129) (820229c) * (DOCSP-32896) Remove mention of <CONTEXT> to reduce damage of prompt leakage (#135) (1484426) * Update internal docs (#134) (a3b2f56) * Handle Snooty Data API tombstones (#133) (a775da4) * (DOCSP-32867): Update QA DB name for sept 1 release (#131) (ce4fa3f) * (DOCSP-32868): only boost 2 results + order results by score (#130) (5230992) * (DOCSP-32856) Add a toggle for streamed vs awaited responses to the demo (#126) (94e8a20) * (DOCSP-32808): k8s production readiness (#125) (2cdf539) * Upgrade azure openai (#128) (6b85e63) * (DOCSP-32759) [UI] Use text instead of a banner for "validate answers" (#124) (302b3c7) * (DOCSP-32604) More helpful max messages error message (#123) (1be4087) * (DOCSP-32254): Include information about latest product versions + minor prompt engineering (#116) (2e40123) * (DOCSP-32657): Remove same IP address validation on conversations (#112) (731ec20) * (DOCSP-31108): Add prod index scripts (#120) (3efe895) * (DOCSP-32807): Add pm2 for running server (#122) (f659e4b) * [Snyk] Security upgrade mongodb from 5.7.0 to 5.8.0 (#121) (c985fef) * Autoscaling fix (#119) (641199a) * Source GIT_COMMIT env variable (#95) (647b00a) * Allow DOP origins in staging (#117) (38a9b66) * (DOCSP-32074): Handle dev center page metadata (#96) (e283121) * Release chat-ui-v0.0.4 (d3bad60) * (DOCSP-32371): Allow for embedded content re-ingest after changes to chunking algorithm (#114) (8b9d68e) * (DOCSP-32279): Update prompts to not mention context when don't know answer (#113) (71ca44b) * (DOCSP-32688) Stop dereferencing spec (#115) (add0dd8) * (DOCSP-31438): Horizontal pod autoscaling (#110) (3c124d4) * fix registry (d52ff7d) * Release chat-ui-v0.0.3 (f8113b7) * Add chat-ui release-it config (731e3d6) * Switch Chatbot to default export (b5db7f0) * Chat UI v0.0.2 (#111) (7e5d684) * (DOCSP-32659): Make server call its own base url if not given a full URL path (#109) (6cdd026) * Allow users to override server URL via props (#108) (708e614) * (DOCSP-32547): Update prod env to use dedicated Azure deployment (#104) (ac93735) * Prod drone fixes (#107) (4df6432) * Update QA server DB (#105) (a12e7c9) * (DOCSP-32452): Dev Center data source remove `<img>` and `<div>` tags + youtube directive (#93) (c5dac2f) * (DOCSP-32596): Set up CORS on the server (#103) (76c9530) * (DOCSP-32104): Do not serve demo site in prod environment (#101) (5f0ea11) * (DOCSP-32247): Fix preprocessed content not added to DB (#100) (57cc24e) * fix handling of openapi specs (#99) (6f9b0e4) * fix hanging promise in async recursive operation (#98) (c974058) * Drone fix (#97) (e54a272) * (DOCSP-32363) [UI] handle cases where the LLM stops mid code example (#85) (9e26c7f) * (DOCSP-32155) [UI] UX Feedback (#81) (c34b246) * (DOCSP-32243) Handle and test pages with page-level code block select (#94) (f6c3f7e) * (DOCSP-32490) [UI] Link to current commit on GitHub (#90) (64dbd0d) * (DOCSP-32242): Handle tabbed Snooty content (#92) (f87f67d) * Restore 'tags' and use 'tagsIn' only internally (#91) (035f018) * (DOCSP-32227): Improve OpenAPI spec ingestion (#88) (f369218) * (DOCSP-31343): Add system diagram and info for ingest (#86) (6e4402b) * (DOCSP-32362): Improve conversation request logging (#89) (b251e0d) * (DOCSP-32253): Add more semantically relevant product names to metadata (#83) (5869f22) * (DOCSP-32434) [UI] Add error text for non-modal input (#87) (5de9282) * (DOCSP-32206): Strip comments from rst -> md (#84) (611a50d) * (DOCSP-32194): Fix drone and k8s for ingest/chat-server staging (#80) (b61c9bc) * Define engines to remove EBADENGINE warning (#82) (f333ef5) * (DOCSP-31440, DOCSP-32075): Add tags to chunk metadata (#78) (254ce2b) * (DOCSP-32217): Change chunker to chunk based on number of tokens + add max context tokens per message (#79) (e7c1cb6) * (DOCSP-32183 & DOCSP-32226): Automate test creation based on YAML file (#77) (30eb660) * Add database index creation scripts (#75) (53f296f) * (DOCSP-31575): LLM qualitative testing framework (#74) (f222e54) * (DOCSP-31111) [UI] publish react component to npm (#73) (3baf40e) * Refactor `ConversationService` class as `makeConversationService` func (#76) (6cb9a83) * (DOCSP-32070): LLM preprocessing on user queries (#72) (6ff4219) * (DOCSP-31925): Support tables in parsed snooty MD (#67) (539d403) * Reduce ChatGPT API flakiness + faster timeout when flaky (#69) (8938599) * Update Chat UI Packaging (#71) (2e5a0fb) * (DOCSP-32066): Q&A server reduce amount of backoff on embedding (#68) (fe938b0) * (DOCSP-31639): fix timeout err in tests (#70) (741ea29) * Implement page titles and prepending frontmatter to chunks (#64) (cf56a26) * Fix JS error surfaced in the UI (#66) (5e6866a) * bundle LG Chat with component lib (c4fc213) * Modify ui config for pre-release (135d211) * (DOCSP-31154) [UI] Handle timeout & "conversation too long" errors (#65) (1943608) * (DOCSP-31791): Boost manual in the results for short queries + make app config file (#63) (6449993) * (DOCSP-31523) [UI] Replace custom components with LG Chat (#61) (3ee640b) * Fix ingest drone + err handling (#59) (8307750) * Fix code block lang (#62) (cffcbf0) * Get current branch from Snooty Data API plus allow override (#56) (6114e7c) * Clean up pages for embedding content (#60) (45ce277) * (DOCSP-31799): Parse definition lists (and other relevant content inside of `"target"` nodes) (#58) (2d7d1b3) * rename chat-ui package (#57) (dc7794f) * (DOCSP-31574): Create qualitative testing env (#55) (0a1c92a) * Add tests for data deletion (#48) (9b1417a) * Update Snooty Data Source to strip 'index' (#50) (d6b0b2f) * Add ability to override Snooty project baseUrl (#52) (fccc8ca) * Split up tests + add utils (#53) (0dedc6f) * (DOCSP-31436) [Q&A] Investigate & improve streaming rate (#54) (cc90cd0) * (DOCSP-31120) [Q&A] separate further reading reference links from LLM response (#43) (a604d83) * (DOCSP-30863) [Q&A] type check requests (#23) (5f908c0) * Streamline/nitpick/bikeshed the PR template (#51) (28b4e1b) * (DOCSP-31601): Update server base URL (#49) (1c60a0b) * (DOCSP-31573): Update prompts to address bugs (#47) (a366803) * (DOCSP-30602): Ingest service prod pipeline (#42) (8e60755) * Update pull_request_template.md (ab166a7) * Monorepo docker build fix (#46) (1dcef28) * (DOCSP-31284): Integrate prod Snooty Data API (#44) (f80ce51) * (DOCSP-31360) [UI] Conditional Streaming (#45) (34490bb) * update root package-lock (578b06b) * rename chat-ui package (d7b6dcf) * Add shared dev script to monorepo root (67811d1) * Streaming works (#25) (c173ba2) * (DOCSP-30601): Create production release system (#41) (7472c9d) * (DOCSP-31304): Implement 'all' command (#40) (f0c4cf2) * (DOCSP-31201) [UI] Updated Legalese (#39) (39c0f97) * (DOCSP-30584): ingest service CI testing and staging drone pipeline (#38) (d2e8325) * Fix + improve devcenter source tests (#37) (be0a5eb) * Fix URL building for devcenter paths (#36) (4c75175) * Add logic for ingest resumption (#35) (0f650c2) * stricter LLM to reduce halucination risk + change demo page title (#33) (cfb4628) * (DOCSP-30890) [UI] Make component mobile responsive (#29) (66080a7) * Add minor optimization for rate-limited embed requests (#34) (7547e31) * (DOCSP-31121): Add request tracking across logs (#32) (7fdc1e5) * (DOCSP-30988): Implement ingest E2E (#28) (8467989) * (DOCSP-30843): IP Address tracking and validation (#21) (7335672) * Update seed-data script (4f7a2f7) * (DOCSP-31050): Fix CI tests + run tests on PR (#30) (74fe090) * (DOCSP-30953 & DOCSP-30945): Error and edge case handling (#24) (af1b845) * Add PR template to repo (#26) (5231ef0) * Fix tests (#27) (58599ea) * more specific linter targets (18d2f5d) * Move and refactor EmbeddedContent/Store and clean up core services (#19) (43078a2) * separate format from lint (3a817b7) * remove console logs (#22) (20027ce) * Add lists to LGMarkdown (#20) (14cf986) * (DOCSP-30964): Staging build (#18) (1163ff6) * Fix renames (945337e) * Rename "chunks" to "embedded content" (#17) (a18bb8d) * Implement Data Sources (#12) (e9c5784) * Move reusable pieces to chat-core (#14) (3babf35) * (DOCSP-30577): POST `/conversations/:conversationId/messages/:messageId/rating` endpoint (#16) (c659173) * E2E (#15) (c85a2aa) * failed message handling (16acff7) * better prompt validation error handling (6408669) * UI tweaks (f803938) * (DOCSP-30921) [UI] Add prompt validation & error handling (1) (cb98ec7) * (DOCSP-30616): `POST /conversations/:conversationId/messages/` route, no streaming (#13) (8451c08) * Add publishing pipeline (42ab17e) * (DOCSP-30575): POST /conversations/ route (#11) (d609171) * (DOCSP-30619, DOCSP-30758, DOCSP-30722): Add MongoDB integration and conversation/content services (#9) (020a501) * Implement ingest store/database (#8) (5b9f8f6) * Add initial design docs (#1) (31d9069) * (DOCSP-30585): Generate search index script (#10) (458403f) * Merge branch 'demodalize' of github.com:nlarew/docs-chatbot (8d46f37) * better avatars (4c7c712) * tweak card heading (40d1dff) * loading ui (58256da) * (DOCSP-30617): OpenAI LLM service (#7) (d4af1f9) * refactor + loading states (4418f8d) * (DOCSP-30618): Create embeddings service (#6) (c54d123) * mock out Q&A API (9d80c20) * Refactor to use a single chat input (6ac351a) * Animation works except for Card ref (2bc2b41) * Rename & refactor (b5385af) * Send button for all inputs with text (9ca1873) * Move input state up (664b456) * Animation works (b8c2f42) * Use card modal WIP (27c2392) * cleanup (d96bcf0) * Fix bad imports (524f896) * (DOCSP-30576): Stub out Q&A server (#4) (899c0a3) * LGMarkdown (33596cf) * Working on LGMarkdown (897b141) * consistent models with backend (319b2ae) * UI lint pass (d8051d9) * Leafie avatar (ebd660b) * Don't require React Context for conversations (86c90b6) * fix modal style (6ae1500) * Add storybook (73a4ec3) * Modal (cac03d2) * fix banner width (088d858) * Merge pull request #3 from mongodb/DOCSP-30565 (5264da5) * Update .drone.yml (a9e3ba5) * working staging build + updated untested prod (fe16f6c) * remove uncreated env secrets (318c501) * fix broken registry path (a1e1dff) * change release (072100f) * update environment info (ff22f4f) * empty state (a66ce15) * fix value file location (ee4f06b) * fix pipeline (89f9e8b) * Trigger build (408d7dc) * Add (broken) tests (329d9e0) * Trigger build (6cae68f) * Fix context (7d213c0) * update path to docker file (f0edac3) * trigger build (e5e64c0) * more debugging (2dfb919) * Use dependency injection for PageStore/ChunkStore interfaces (89577b8) * Remove start comment (287182d) * move drone file (7453f99) * trigger build? (95579f6) * test deploy pipeline (aab4632) * add cta (db01cdb) * apply prettier (c49d368) * remove unused imports (85f6ab3) * add prettier (c1cf166) * combined props (ae9d66c) * chat state works (e29ee25) * add temp aria-labelledby prop (a0d4695) * Scaffold the ingest CLI (81a5c41) * Update ingest/README (65d2902) * chatbot-ui -> chat-ui (4340d1a) * Merge pull request #2 from mongodben/DOCSP-30486 (17662e6) * Bootstrap ingest CLI tool (0590da2) * Add chatbot-ui (997bec5) * set up QA server project structure (7cd25d5) * Delete specs directory (6b06d68) * ready repo for first push (2b250da) * refactor chatbot server to use latest versions * remove trigger files * clean drone up after changes for development * update contributor guide * Update CONTRIBUTING.md Co-authored-by: Chris Bush <chris.bush@mongodb.com> * Update chat-core/src/extractFrontMatter.ts --------- Co-authored-by: Nick Larew <nick.larew@mongodb.com> Co-authored-by: Chris Bush <chris.bush@mongodb.com> --------- Co-authored-by: Chris Bush <chris.bush@mongodb.com> Co-authored-by: Nick Larew <nick.larew@mongodb.com>

* (EAI-28): Ingest MongoDB University transcript content (#247) (8d7bed9) * Release chat-ui v0.1.2 (c7edbdd) * Upgrade lockfile (90c2a15) * (EAI-145) InputBar adds newlines on shift+enter (#248) (b6f6d84) * (EAI-150) [UI] change "Further Reading" in chatbot response to "Related Resources" (#249) (401771e) * Create sampleMdxFile.mdx (802a614) * (EAI-138): Refactor origin handling, add custom conversation data logic, add conversations middleware (#244) (ce35576) * Deploy docs to https://mongodb.github.io/chatbot/ (#246) (af8ece8) * (EAI-18): Docs for RAG framework (#243) (ef297b2) * Release chat-ui v0.1.1 (f612ffb) * Update exports & format (d159105) * Ubiquitous dark mode (af7c8b8) * Remove unused props & update README (9341493) * Release chat-ui v0.1.0 (8a1853b) * Use a more specific env variable name (f8f4527) * Release chat-ui v0.1.0-0 (65078b7) * Remove @release-it/bumper from chat-ui (9052701) * Remove pending github token (38966a7) * Update pull_request_template.md to use new Jira project (810cb5c) * (EAI-7) [UI] Can't send existing text after reopening modal (#242) (2fbe68f) * (EAI-141): Add link to dev center blog post in README (#241) (6ab06a2) * Initial OSS Refactor (#208) (bba6322) * [Snyk] Upgrade mongodb from 5.8.1 to 5.9.1 (#238) (be8f145) * (EAI-137) [Q&A] Custom request origins (#237) (7d08d4e) * Artifactory auth with token instead of username/password (#239) (a6d1521) * (DOCSP-33030) [Q&A] Record request origin for usage tracking (#234) (eb13f4b) * (EAI-123) [UI] Replace chat-core with mongodb-rag-core (#236) (2afbdce) * (DOCSP-33938) [UI] Fix nested newline spacing (#233) (a879642) * (DOCSP-34007) [UI] Font size & spacing issues on live docs site (#235) (7b4bd3c) * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) (b8541a7) * (DOCSP-32603): Add Terraform data source (#205) (0d64743) * dev QoL scripts (3813ad7) * (DOCSP-33744): update QA server DB name for latest snapshot (#210) (5df2bf0) * (DOCSP-32482): Build out qualitative tests (#201) (0535a89) * Conversations refactor (#199) (5a60a15) * Release ingest-v0.1.2 (#204) (ed34507) * [Snyk] Upgrade urllib from 3.19.1 to 3.19.2 (#203) (743cc01) * Add Rust and Entity Framework Snooty data sources (#202) (ff15139) * [Snyk] Upgrade urllib from 3.19.0 to 3.19.1 (#200) (dea05e0) * Release chat-server v0.1.3 (ac542c8) * Use GITHUB_TOKEN for chat-ui releases (38a9e1a) * [Snyk] Upgrade urllib from 3.18.1 to 3.19.0 (#197) (dd2baaf) * Do not fail query on rephrase fail (#196) (8b7c60a) * Release chat-ui v0.0.8 (03bf7cb) * Bundle new @lg-chat components (2f1f11b) * Release chat-ui v0.0.7 (d082a63) * Unify release-it config (419422e) * Release chat-server v0.1.2 (b0b425c) * [Snyk] Upgrade urllib from 3.18.0 to 3.18.1 (#194) (2c98ca4) * [DOCSP-32950] Create a DevCenter Chatbot Component (#193) (c219d03) * (DOCSP-33445) [ANALYTICS] Scrubbed Views (#195) (8ef2bd7) * (DOCSP-33419) [ANALYTICS] Operational Views (#192) (aae8259) * Release scripts v0.1.2 (fbf2bbf) * Add whether message was deemed "Do not answer" to conversation (#191) (955b722) * (DOCSP-31104) [UI] Add tests to FE services (#190) (d3e2000) * Release ingest v0.1.1 (5b06cd2) * Update package-lock.json (a568d00) * Replace GitHub data source (#188) (747d08e) * Update env vars for analyze script (0a46085) * Add scrubbed message analyzer (#187) (9cef6b8) * (DOCSP-33291) [UI] Add tck tracking to links (#185) (acfb6a1) * Update README.md (a38cbf7) * Release 0.1.1 (8ee27c1) * Fix typo in scripts build (0534458) * Release 0.1.0 (09fa262) * Add release-it to scripts (9ea2e71) * Trigger drone run for scripts service (b1ffcc8) * Fix drone scripts (#186) (9bae8e9) * Cron scrub (#184) (d67dda9) * (DOCSP-33273) Open Source Licensing (#183) (00611bf) * De-flake SnootyProjectsInfo test (#181) (9f7250e) * (DOCSP-32930): Add pymongo metadata (#179) (6694209) * Update release-it git add config to add monorepo stuff (679e447) * Update package-lock in monorepo for release (e7308ef) * Release 0.1.1 (d60d14b) * Update release-it pre-hooks (718b96d) * Run npm audit fix (#178) (c74e1a9) * Remove `seed-content` package (#157) (1b9ac43) * Add scrub message script (#177) (ee2f864) * Remove extra padding around initial input (9dc27ad) * (DOCSP-33094): Save conversation embeddings (#175) (9c6e79d) * Ingest release 0.1.0 (#174) (4928eed) * Release chat-ui-v0.0.6 (2bf2a0e) * (DOCSP-33059) [UI] Fix suggested prompts & other LG Chat tweaks (#171) (cde1b74) * chat server v0.1.0 (#170) (ba69ad8) * new commit for build (56ed01e) * trigger ingest build (4052af9) * clean up spec + remove ip address validation from rate msg (#166) (28cf222) * UI Tweaks (#159) (53b392f) * Delete chat-server/trigger.me (#165) (5542f41) * update repo name in code (#163) (f9628bb) * Update README.md (5ad9cd0) * Launch pre releases (#162) (9b2380b) * Expand Contributing Guide and `ingest` README (#161) (30434e3) * filter out very small chunks (#160) (d952c6d) * (DOCSP-31928): Implement numbered lists (#154) (f41258d) * clarify ingest readme (115b00b) * move git install to correct docker stage (40600cb) * (DOCSP-32991, DOCSP-32992): `GitDataSource`, functionality for converting HTML to Markdown, ingest Scala, Java Reactive Streams, and C docs (#152) (7118444) * (DOCSP-32915) [UI] Streamer bugs out when switching tabs (#156) (91c288d) * (DOCSP-33090) [UI] Remove experimental badge from input on focus (#155) (a3ea236) * (DOCSP-32993, DOCSP-32906, DOCSP-32993, DOCSP-32990): Data sources for MD on Github + data ingestion of related content (#148) (7daed63) * change recommended prompt (#150) (2521f08) * add missing env example (#153) (1201a34) * [Snyk] Upgrade mongodb from 5.8.0 to 5.8.1 (#151) (966a4bf) * make preprocessor less biased toward shell (#149) (886cbe5) * (DOCSP-32616): Refactor config, add library, and log config (#143) (4fcfc58) * add default + remove unused style (0296b42) * Update chat-ui release config (3174b61) * Release chat-ui-v0.0.5 (65727f7) * Add root release:chat-ui command (4621989) * (DOCSP-32757) [UI] Update to Responsive LG Chat Components (#145) (81b70bb) * (DOCSP-32670): Basic server load testing (#136) (282c947) * [Snyk] Upgrade urllib from 3.17.2 to 3.18.0 (#147) (aefa9e0) * (DOCSP-32756): Implement headings in tables (#140) (ea416ce) * custom/synthetic/meta data source content (#127) (d687312) * [Snyk] Upgrade urllib from 3.17.1 to 3.17.2 (#144) (209135a) * (DOCSP-32673) [UI] Suppress focus to prevent modal reopen (#139) (43b8a98) * (DOCSP-32936): Increase single IP rate limits to be more line w firewall rate limits (#141) (87d8785) * (DOCSP-31863): Fix flaky test, refactor core llm service + server implementation (#142) (584a5e3) * (DOCSP-32916): Conversation service rate limiting + slow down (#138) (a88a5d5) * (DOCSP-32914): Stop unindenting context (#137) (27630c6) * (DOCSP-31285): Ingest PyMongo docs (#118) (e04c10b) * (DOCSP-32694) Dark mode: use LG palette & provider + cleanup (a95f5c5) * (DOCSP-32694) [UI] Add darkMode prop (#129) (820229c) * (DOCSP-32896) Remove mention of <CONTEXT> to reduce damage of prompt leakage (#135) (1484426) * Update internal docs (#134) (a3b2f56) * Handle Snooty Data API tombstones (#133) (a775da4) * (DOCSP-32867): Update QA DB name for sept 1 release (#131) (ce4fa3f) * (DOCSP-32868): only boost 2 results + order results by score (#130) (5230992) * (DOCSP-32856) Add a toggle for streamed vs awaited responses to the demo (#126) (94e8a20) * (DOCSP-32808): k8s production readiness (#125) (2cdf539) * Upgrade azure openai (#128) (6b85e63) * (DOCSP-32759) [UI] Use text instead of a banner for "validate answers" (#124) (302b3c7) * (DOCSP-32604) More helpful max messages error message (#123) (1be4087) * (DOCSP-32254): Include information about latest product versions + minor prompt engineering (#116) (2e40123) * (DOCSP-32657): Remove same IP address validation on conversations (#112) (731ec20) * (DOCSP-31108): Add prod index scripts (#120) (3efe895) * (DOCSP-32807): Add pm2 for running server (#122) (f659e4b) * [Snyk] Security upgrade mongodb from 5.7.0 to 5.8.0 (#121) (c985fef) * Autoscaling fix (#119) (641199a) * Source GIT_COMMIT env variable (#95) (647b00a) * Allow DOP origins in staging (#117) (38a9b66) * (DOCSP-32074): Handle dev center page metadata (#96) (e283121) * Release chat-ui-v0.0.4 (d3bad60) * (DOCSP-32371): Allow for embedded content re-ingest after changes to chunking algorithm (#114) (8b9d68e) * (DOCSP-32279): Update prompts to not mention context when don't know answer (#113) (71ca44b) * (DOCSP-32688) Stop dereferencing spec (#115) (add0dd8) * (DOCSP-31438): Horizontal pod autoscaling (#110) (3c124d4) * fix registry (d52ff7d) * Release chat-ui-v0.0.3 (f8113b7) * Add chat-ui release-it config (731e3d6) * Switch Chatbot to default export (b5db7f0) * Chat UI v0.0.2 (#111) (7e5d684) * (DOCSP-32659): Make server call its own base url if not given a full URL path (#109) (6cdd026) * Allow users to override server URL via props (#108) (708e614) * (DOCSP-32547): Update prod env to use dedicated Azure deployment (#104) (ac93735) * Prod drone fixes (#107) (4df6432) * Update QA server DB (#105) (a12e7c9) * (DOCSP-32452): Dev Center data source remove `<img>` and `<div>` tags + youtube directive (#93) (c5dac2f) * (DOCSP-32596): Set up CORS on the server (#103) (76c9530) * (DOCSP-32104): Do not serve demo site in prod environment (#101) (5f0ea11) * (DOCSP-32247): Fix preprocessed content not added to DB (#100) (57cc24e) * fix handling of openapi specs (#99) (6f9b0e4) * fix hanging promise in async recursive operation (#98) (c974058) * Drone fix (#97) (e54a272) * (DOCSP-32363) [UI] handle cases where the LLM stops mid code example (#85) (9e26c7f) * (DOCSP-32155) [UI] UX Feedback (#81) (c34b246) * (DOCSP-32243) Handle and test pages with page-level code block select (#94) (f6c3f7e) * (DOCSP-32490) [UI] Link to current commit on GitHub (#90) (64dbd0d) * (DOCSP-32242): Handle tabbed Snooty content (#92) (f87f67d) * Restore 'tags' and use 'tagsIn' only internally (#91) (035f018) * (DOCSP-32227): Improve OpenAPI spec ingestion (#88) (f369218) * (DOCSP-31343): Add system diagram and info for ingest (#86) (6e4402b) * (DOCSP-32362): Improve conversation request logging (#89) (b251e0d) * (DOCSP-32253): Add more semantically relevant product names to metadata (#83) (5869f22) * (DOCSP-32434) [UI] Add error text for non-modal input (#87) (5de9282) * (DOCSP-32206): Strip comments from rst -> md (#84) (611a50d) * (DOCSP-32194): Fix drone and k8s for ingest/chat-server staging (#80) (b61c9bc) * Define engines to remove EBADENGINE warning (#82) (f333ef5) * (DOCSP-31440, DOCSP-32075): Add tags to chunk metadata (#78) (254ce2b) * (DOCSP-32217): Change chunker to chunk based on number of tokens + add max context tokens per message (#79) (e7c1cb6) * (DOCSP-32183 & DOCSP-32226): Automate test creation based on YAML file (#77) (30eb660) * Add database index creation scripts (#75) (53f296f) * (DOCSP-31575): LLM qualitative testing framework (#74) (f222e54) * (DOCSP-31111) [UI] publish react component to npm (#73) (3baf40e) * Refactor `ConversationService` class as `makeConversationService` func (#76) (6cb9a83) * (DOCSP-32070): LLM preprocessing on user queries (#72) (6ff4219) * (DOCSP-31925): Support tables in parsed snooty MD (#67) (539d403) * Reduce ChatGPT API flakiness + faster timeout when flaky (#69) (8938599) * Update Chat UI Packaging (#71) (2e5a0fb) * (DOCSP-32066): Q&A server reduce amount of backoff on embedding (#68) (fe938b0) * (DOCSP-31639): fix timeout err in tests (#70) (741ea29) * Implement page titles and prepending frontmatter to chunks (#64) (cf56a26) * Fix JS error surfaced in the UI (#66) (5e6866a) * bundle LG Chat with component lib (c4fc213) * Modify ui config for pre-release (135d211) * (DOCSP-31154) [UI] Handle timeout & "conversation too long" errors (#65) (1943608) * (DOCSP-31791): Boost manual in the results for short queries + make app config file (#63) (6449993) * (DOCSP-31523) [UI] Replace custom components with LG Chat (#61) (3ee640b) * Fix ingest drone + err handling (#59) (8307750) * Fix code block lang (#62) (cffcbf0) * Get current branch from Snooty Data API plus allow override (#56) (6114e7c) * Clean up pages for embedding content (#60) (45ce277) * (DOCSP-31799): Parse definition lists (and other relevant content inside of `"target"` nodes) (#58) (2d7d1b3) * rename chat-ui package (#57) (dc7794f) * (DOCSP-31574): Create qualitative testing env (#55) (0a1c92a) * Add tests for data deletion (#48) (9b1417a) * Update Snooty Data Source to strip 'index' (#50) (d6b0b2f) * Add ability to override Snooty project baseUrl (#52) (fccc8ca) * Split up tests + add utils (#53) (0dedc6f) * (DOCSP-31436) [Q&A] Investigate & improve streaming rate (#54) (cc90cd0) * (DOCSP-31120) [Q&A] separate further reading reference links from LLM response (#43) (a604d83) * (DOCSP-30863) [Q&A] type check requests (#23) (5f908c0) * Streamline/nitpick/bikeshed the PR template (#51) (28b4e1b) * (DOCSP-31601): Update server base URL (#49) (1c60a0b) * (DOCSP-31573): Update prompts to address bugs (#47) (a366803) * (DOCSP-30602): Ingest service prod pipeline (#42) (8e60755) * Update pull_request_template.md (ab166a7) * Monorepo docker build fix (#46) (1dcef28) * (DOCSP-31284): Integrate prod Snooty Data API (#44) (f80ce51) * (DOCSP-31360) [UI] Conditional Streaming (#45) (34490bb) * update root package-lock (578b06b) * rename chat-ui package (d7b6dcf) * Add shared dev script to monorepo root (67811d1) * Streaming works (#25) (c173ba2) * (DOCSP-30601): Create production release system (#41) (7472c9d) * (DOCSP-31304): Implement 'all' command (#40) (f0c4cf2) * (DOCSP-31201) [UI] Updated Legalese (#39) (39c0f97) * (DOCSP-30584): ingest service CI testing and staging drone pipeline (#38) (d2e8325) * Fix + improve devcenter source tests (#37) (be0a5eb) * Fix URL building for devcenter paths (#36) (4c75175) * Add logic for ingest resumption (#35) (0f650c2) * stricter LLM to reduce halucination risk + change demo page title (#33) (cfb4628) * (DOCSP-30890) [UI] Make component mobile responsive (#29) (66080a7) * Add minor optimization for rate-limited embed requests (#34) (7547e31) * (DOCSP-31121): Add request tracking across logs (#32) (7fdc1e5) * (DOCSP-30988): Implement ingest E2E (#28) (8467989) * (DOCSP-30843): IP Address tracking and validation (#21) (7335672) * Update seed-data script (4f7a2f7) * (DOCSP-31050): Fix CI tests + run tests on PR (#30) (74fe090) * (DOCSP-30953 & DOCSP-30945): Error and edge case handling (#24) (af1b845) * Add PR template to repo (#26) (5231ef0) * Fix tests (#27) (58599ea) * more specific linter targets (18d2f5d) * Move and refactor EmbeddedContent/Store and clean up core services (#19) (43078a2) * separate format from lint (3a817b7) * remove console logs (#22) (20027ce) * Add lists to LGMarkdown (#20) (14cf986) * (DOCSP-30964): Staging build (#18) (1163ff6) * Fix renames (945337e) * Rename "chunks" to "embedded content" (#17) (a18bb8d) * Implement Data Sources (#12) (e9c5784) * Move reusable pieces to chat-core (#14) (3babf35) * (DOCSP-30577): POST `/conversations/:conversationId/messages/:messageId/rating` endpoint (#16) (c659173) * E2E (#15) (c85a2aa) * failed message handling (16acff7) * better prompt validation error handling (6408669) * UI tweaks (f803938) * (DOCSP-30921) [UI] Add prompt validation & error handling (1) (cb98ec7) * (DOCSP-30616): `POST /conversations/:conversationId/messages/` route, no streaming (#13) (8451c08) * Add publishing pipeline (42ab17e) * (DOCSP-30575): POST /conversations/ route (#11) (d609171) * (DOCSP-30619, DOCSP-30758, DOCSP-30722): Add MongoDB integration and conversation/content services (#9) (020a501) * Implement ingest store/database (#8) (5b9f8f6) * Add initial design docs (#1) (31d9069) * (DOCSP-30585): Generate search index script (#10) (458403f) * Merge branch 'demodalize' of github.com:nlarew/docs-chatbot (8d46f37) * better avatars (4c7c712) * tweak card heading (40d1dff) * loading ui (58256da) * (DOCSP-30617): OpenAI LLM service (#7) (d4af1f9) * refactor + loading states (4418f8d) * (DOCSP-30618): Create embeddings service (#6) (c54d123) * mock out Q&A API (9d80c20) * Refactor to use a single chat input (6ac351a) * Animation works except for Card ref (2bc2b41) * Rename & refactor (b5385af) * Send button for all inputs with text (9ca1873) * Move input state up (664b456) * Animation works (b8c2f42) * Use card modal WIP (27c2392) * cleanup (d96bcf0) * Fix bad imports (524f896) * (DOCSP-30576): Stub out Q&A server (#4) (899c0a3) * LGMarkdown (33596cf) * Working on LGMarkdown (897b141) * consistent models with backend (319b2ae) * UI lint pass (d8051d9) * Leafie avatar (ebd660b) * Don't require React Context for conversations (86c90b6) * fix modal style (6ae1500) * Add storybook (73a4ec3) * Modal (cac03d2) * fix banner width (088d858) * Merge pull request #3 from mongodb/DOCSP-30565 (5264da5) * Update .drone.yml (a9e3ba5) * working staging build + updated untested prod (fe16f6c) * remove uncreated env secrets (318c501) * fix broken registry path (a1e1dff) * change release (072100f) * update environment info (ff22f4f) * empty state (a66ce15) * fix value file location (ee4f06b) * fix pipeline (89f9e8b) * Trigger build (408d7dc) * Add (broken) tests (329d9e0) * Trigger build (6cae68f) * Fix context (7d213c0) * update path to docker file (f0edac3) * trigger build (e5e64c0) * more debugging (2dfb919) * Use dependency injection for PageStore/ChunkStore interfaces (89577b8) * Remove start comment (287182d) * move drone file (7453f99) * trigger build? (95579f6) * test deploy pipeline (aab4632) * add cta (db01cdb) * apply prettier (c49d368) * remove unused imports (85f6ab3) * add prettier (c1cf166) * combined props (ae9d66c) * chat state works (e29ee25) * add temp aria-labelledby prop (a0d4695) * Scaffold the ingest CLI (81a5c41) * Update ingest/README (65d2902) * chatbot-ui -> chat-ui (4340d1a) * Merge pull request #2 from mongodben/DOCSP-30486 (17662e6) * Bootstrap ingest CLI tool (0590da2) * Add chatbot-ui (997bec5) * set up QA server project structure (7cd25d5) * Delete specs directory (6b06d68) * ready repo for first push (2b250da)

* Fix broken CI (#253) (7e44368) * Release mongodb-chatbot-server v0.2.0 (#252) (7dbd308) * (EAI-28): Ingest MongoDB University transcript content (#247) (8d7bed9) * Release chat-ui v0.1.2 (c7edbdd) * Upgrade lockfile (90c2a15) * (EAI-145) InputBar adds newlines on shift+enter (#248) (b6f6d84) * (EAI-150) [UI] change "Further Reading" in chatbot response to "Related Resources" (#249) (401771e) * Create sampleMdxFile.mdx (802a614) * (EAI-138): Refactor origin handling, add custom conversation data logic, add conversations middleware (#244) (ce35576) * Deploy docs to https://mongodb.github.io/chatbot/ (#246) (af8ece8) * (EAI-18): Docs for RAG framework (#243) (ef297b2) * Release chat-ui v0.1.1 (f612ffb) * Update exports & format (d159105) * Ubiquitous dark mode (af7c8b8) * Remove unused props & update README (9341493) * Release chat-ui v0.1.0 (8a1853b) * Use a more specific env variable name (f8f4527) * Release chat-ui v0.1.0-0 (65078b7) * Remove @release-it/bumper from chat-ui (9052701) * Remove pending github token (38966a7) * Update pull_request_template.md to use new Jira project (810cb5c) * (EAI-7) [UI] Can't send existing text after reopening modal (#242) (2fbe68f) * (EAI-141): Add link to dev center blog post in README (#241) (6ab06a2) * Initial OSS Refactor (#208) (bba6322) * [Snyk] Upgrade mongodb from 5.8.1 to 5.9.1 (#238) (be8f145) * (EAI-137) [Q&A] Custom request origins (#237) (7d08d4e) * Artifactory auth with token instead of username/password (#239) (a6d1521) * (DOCSP-33030) [Q&A] Record request origin for usage tracking (#234) (eb13f4b) * (EAI-123) [UI] Replace chat-core with mongodb-rag-core (#236) (2afbdce) * (DOCSP-33938) [UI] Fix nested newline spacing (#233) (a879642) * (DOCSP-34007) [UI] Font size & spacing issues on live docs site (#235) (7b4bd3c) * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) (b8541a7) * (DOCSP-32603): Add Terraform data source (#205) (0d64743) * dev QoL scripts (3813ad7) * (DOCSP-33744): update QA server DB name for latest snapshot (#210) (5df2bf0) * (DOCSP-32482): Build out qualitative tests (#201) (0535a89) * Conversations refactor (#199) (5a60a15) * Release ingest-v0.1.2 (#204) (ed34507) * [Snyk] Upgrade urllib from 3.19.1 to 3.19.2 (#203) (743cc01) * Add Rust and Entity Framework Snooty data sources (#202) (ff15139) * [Snyk] Upgrade urllib from 3.19.0 to 3.19.1 (#200) (dea05e0) * Release chat-server v0.1.3 (ac542c8) * Use GITHUB_TOKEN for chat-ui releases (38a9e1a) * [Snyk] Upgrade urllib from 3.18.1 to 3.19.0 (#197) (dd2baaf) * Do not fail query on rephrase fail (#196) (8b7c60a) * Release chat-ui v0.0.8 (03bf7cb) * Bundle new @lg-chat components (2f1f11b) * Release chat-ui v0.0.7 (d082a63) * Unify release-it config (419422e) * Release chat-server v0.1.2 (b0b425c) * [Snyk] Upgrade urllib from 3.18.0 to 3.18.1 (#194) (2c98ca4) * [DOCSP-32950] Create a DevCenter Chatbot Component (#193) (c219d03) * (DOCSP-33445) [ANALYTICS] Scrubbed Views (#195) (8ef2bd7) * (DOCSP-33419) [ANALYTICS] Operational Views (#192) (aae8259) * Release scripts v0.1.2 (fbf2bbf) * Add whether message was deemed "Do not answer" to conversation (#191) (955b722) * (DOCSP-31104) [UI] Add tests to FE services (#190) (d3e2000) * Release ingest v0.1.1 (5b06cd2) * Update package-lock.json (a568d00) * Replace GitHub data source (#188) (747d08e) * Update env vars for analyze script (0a46085) * Add scrubbed message analyzer (#187) (9cef6b8) * (DOCSP-33291) [UI] Add tck tracking to links (#185) (acfb6a1) * Update README.md (a38cbf7) * Release 0.1.1 (8ee27c1) * Fix typo in scripts build (0534458) * Release 0.1.0 (09fa262) * Add release-it to scripts (9ea2e71) * Trigger drone run for scripts service (b1ffcc8) * Fix drone scripts (#186) (9bae8e9) * Cron scrub (#184) (d67dda9) * (DOCSP-33273) Open Source Licensing (#183) (00611bf) * De-flake SnootyProjectsInfo test (#181) (9f7250e) * (DOCSP-32930): Add pymongo metadata (#179) (6694209) * Update release-it git add config to add monorepo stuff (679e447) * Update package-lock in monorepo for release (e7308ef) * Release 0.1.1 (d60d14b) * Update release-it pre-hooks (718b96d) * Run npm audit fix (#178) (c74e1a9) * Remove `seed-content` package (#157) (1b9ac43) * Add scrub message script (#177) (ee2f864) * Remove extra padding around initial input (9dc27ad) * (DOCSP-33094): Save conversation embeddings (#175) (9c6e79d) * Ingest release 0.1.0 (#174) (4928eed) * Release chat-ui-v0.0.6 (2bf2a0e) * (DOCSP-33059) [UI] Fix suggested prompts & other LG Chat tweaks (#171) (cde1b74) * chat server v0.1.0 (#170) (ba69ad8) * new commit for build (56ed01e) * trigger ingest build (4052af9) * clean up spec + remove ip address validation from rate msg (#166) (28cf222) * UI Tweaks (#159) (53b392f) * Delete chat-server/trigger.me (#165) (5542f41) * update repo name in code (#163) (f9628bb) * Update README.md (5ad9cd0) * Launch pre releases (#162) (9b2380b) * Expand Contributing Guide and `ingest` README (#161) (30434e3) * filter out very small chunks (#160) (d952c6d) * (DOCSP-31928): Implement numbered lists (#154) (f41258d) * clarify ingest readme (115b00b) * move git install to correct docker stage (40600cb) * (DOCSP-32991, DOCSP-32992): `GitDataSource`, functionality for converting HTML to Markdown, ingest Scala, Java Reactive Streams, and C docs (#152) (7118444) * (DOCSP-32915) [UI] Streamer bugs out when switching tabs (#156) (91c288d) * (DOCSP-33090) [UI] Remove experimental badge from input on focus (#155) (a3ea236) * (DOCSP-32993, DOCSP-32906, DOCSP-32993, DOCSP-32990): Data sources for MD on Github + data ingestion of related content (#148) (7daed63) * change recommended prompt (#150) (2521f08) * add missing env example (#153) (1201a34) * [Snyk] Upgrade mongodb from 5.8.0 to 5.8.1 (#151) (966a4bf) * make preprocessor less biased toward shell (#149) (886cbe5) * (DOCSP-32616): Refactor config, add library, and log config (#143) (4fcfc58) * add default + remove unused style (0296b42) * Update chat-ui release config (3174b61) * Release chat-ui-v0.0.5 (65727f7) * Add root release:chat-ui command (4621989) * (DOCSP-32757) [UI] Update to Responsive LG Chat Components (#145) (81b70bb) * (DOCSP-32670): Basic server load testing (#136) (282c947) * [Snyk] Upgrade urllib from 3.17.2 to 3.18.0 (#147) (aefa9e0) * (DOCSP-32756): Implement headings in tables (#140) (ea416ce) * custom/synthetic/meta data source content (#127) (d687312) * [Snyk] Upgrade urllib from 3.17.1 to 3.17.2 (#144) (209135a) * (DOCSP-32673) [UI] Suppress focus to prevent modal reopen (#139) (43b8a98) * (DOCSP-32936): Increase single IP rate limits to be more line w firewall rate limits (#141) (87d8785) * (DOCSP-31863): Fix flaky test, refactor core llm service + server implementation (#142) (584a5e3) * (DOCSP-32916): Conversation service rate limiting + slow down (#138) (a88a5d5) * (DOCSP-32914): Stop unindenting context (#137) (27630c6) * (DOCSP-31285): Ingest PyMongo docs (#118) (e04c10b) * (DOCSP-32694) Dark mode: use LG palette & provider + cleanup (a95f5c5) * (DOCSP-32694) [UI] Add darkMode prop (#129) (820229c) * (DOCSP-32896) Remove mention of <CONTEXT> to reduce damage of prompt leakage (#135) (1484426) * Update internal docs (#134) (a3b2f56) * Handle Snooty Data API tombstones (#133) (a775da4) * (DOCSP-32867): Update QA DB name for sept 1 release (#131) (ce4fa3f) * (DOCSP-32868): only boost 2 results + order results by score (#130) (5230992) * (DOCSP-32856) Add a toggle for streamed vs awaited responses to the demo (#126) (94e8a20) * (DOCSP-32808): k8s production readiness (#125) (2cdf539) * Upgrade azure openai (#128) (6b85e63) * (DOCSP-32759) [UI] Use text instead of a banner for "validate answers" (#124) (302b3c7) * (DOCSP-32604) More helpful max messages error message (#123) (1be4087) * (DOCSP-32254): Include information about latest product versions + minor prompt engineering (#116) (2e40123) * (DOCSP-32657): Remove same IP address validation on conversations (#112) (731ec20) * (DOCSP-31108): Add prod index scripts (#120) (3efe895) * (DOCSP-32807): Add pm2 for running server (#122) (f659e4b) * [Snyk] Security upgrade mongodb from 5.7.0 to 5.8.0 (#121) (c985fef) * Autoscaling fix (#119) (641199a) * Source GIT_COMMIT env variable (#95) (647b00a) * Allow DOP origins in staging (#117) (38a9b66) * (DOCSP-32074): Handle dev center page metadata (#96) (e283121) * Release chat-ui-v0.0.4 (d3bad60) * (DOCSP-32371): Allow for embedded content re-ingest after changes to chunking algorithm (#114) (8b9d68e) * (DOCSP-32279): Update prompts to not mention context when don't know answer (#113) (71ca44b) * (DOCSP-32688) Stop dereferencing spec (#115) (add0dd8) * (DOCSP-31438): Horizontal pod autoscaling (#110) (3c124d4) * fix registry (d52ff7d) * Release chat-ui-v0.0.3 (f8113b7) * Add chat-ui release-it config (731e3d6) * Switch Chatbot to default export (b5db7f0) * Chat UI v0.0.2 (#111) (7e5d684) * (DOCSP-32659): Make server call its own base url if not given a full URL path (#109) (6cdd026) * Allow users to override server URL via props (#108) (708e614) * (DOCSP-32547): Update prod env to use dedicated Azure deployment (#104) (ac93735) * Prod drone fixes (#107) (4df6432) * Update QA server DB (#105) (a12e7c9) * (DOCSP-32452): Dev Center data source remove `<img>` and `<div>` tags + youtube directive (#93) (c5dac2f) * (DOCSP-32596): Set up CORS on the server (#103) (76c9530) * (DOCSP-32104): Do not serve demo site in prod environment (#101) (5f0ea11) * (DOCSP-32247): Fix preprocessed content not added to DB (#100) (57cc24e) * fix handling of openapi specs (#99) (6f9b0e4) * fix hanging promise in async recursive operation (#98) (c974058) * Drone fix (#97) (e54a272) * (DOCSP-32363) [UI] handle cases where the LLM stops mid code example (#85) (9e26c7f) * (DOCSP-32155) [UI] UX Feedback (#81) (c34b246) * (DOCSP-32243) Handle and test pages with page-level code block select (#94) (f6c3f7e) * (DOCSP-32490) [UI] Link to current commit on GitHub (#90) (64dbd0d) * (DOCSP-32242): Handle tabbed Snooty content (#92) (f87f67d) * Restore 'tags' and use 'tagsIn' only internally (#91) (035f018) * (DOCSP-32227): Improve OpenAPI spec ingestion (#88) (f369218) * (DOCSP-31343): Add system diagram and info for ingest (#86) (6e4402b) * (DOCSP-32362): Improve conversation request logging (#89) (b251e0d) * (DOCSP-32253): Add more semantically relevant product names to metadata (#83) (5869f22) * (DOCSP-32434) [UI] Add error text for non-modal input (#87) (5de9282) * (DOCSP-32206): Strip comments from rst -> md (#84) (611a50d) * (DOCSP-32194): Fix drone and k8s for ingest/chat-server staging (#80) (b61c9bc) * Define engines to remove EBADENGINE warning (#82) (f333ef5) * (DOCSP-31440, DOCSP-32075): Add tags to chunk metadata (#78) (254ce2b) * (DOCSP-32217): Change chunker to chunk based on number of tokens + add max context tokens per message (#79) (e7c1cb6) * (DOCSP-32183 & DOCSP-32226): Automate test creation based on YAML file (#77) (30eb660) * Add database index creation scripts (#75) (53f296f) * (DOCSP-31575): LLM qualitative testing framework (#74) (f222e54) * (DOCSP-31111) [UI] publish react component to npm (#73) (3baf40e) * Refactor `ConversationService` class as `makeConversationService` func (#76) (6cb9a83) * (DOCSP-32070): LLM preprocessing on user queries (#72) (6ff4219) * (DOCSP-31925): Support tables in parsed snooty MD (#67) (539d403) * Reduce ChatGPT API flakiness + faster timeout when flaky (#69) (8938599) * Update Chat UI Packaging (#71) (2e5a0fb) * (DOCSP-32066): Q&A server reduce amount of backoff on embedding (#68) (fe938b0) * (DOCSP-31639): fix timeout err in tests (#70) (741ea29) * Implement page titles and prepending frontmatter to chunks (#64) (cf56a26) * Fix JS error surfaced in the UI (#66) (5e6866a) * bundle LG Chat with component lib (c4fc213) * Modify ui config for pre-release (135d211) * (DOCSP-31154) [UI] Handle timeout & "conversation too long" errors (#65) (1943608) * (DOCSP-31791): Boost manual in the results for short queries + make app config file (#63) (6449993) * (DOCSP-31523) [UI] Replace custom components with LG Chat (#61) (3ee640b) * Fix ingest drone + err handling (#59) (8307750) * Fix code block lang (#62) (cffcbf0) * Get current branch from Snooty Data API plus allow override (#56) (6114e7c) * Clean up pages for embedding content (#60) (45ce277) * (DOCSP-31799): Parse definition lists (and other relevant content inside of `"target"` nodes) (#58) (2d7d1b3) * rename chat-ui package (#57) (dc7794f) * (DOCSP-31574): Create qualitative testing env (#55) (0a1c92a) * Add tests for data deletion (#48) (9b1417a) * Update Snooty Data Source to strip 'index' (#50) (d6b0b2f) * Add ability to override Snooty project baseUrl (#52) (fccc8ca) * Split up tests + add utils (#53) (0dedc6f) * (DOCSP-31436) [Q&A] Investigate & improve streaming rate (#54) (cc90cd0) * (DOCSP-31120) [Q&A] separate further reading reference links from LLM response (#43) (a604d83) * (DOCSP-30863) [Q&A] type check requests (#23) (5f908c0) * Streamline/nitpick/bikeshed the PR template (#51) (28b4e1b) * (DOCSP-31601): Update server base URL (#49) (1c60a0b) * (DOCSP-31573): Update prompts to address bugs (#47) (a366803) * (DOCSP-30602): Ingest service prod pipeline (#42) (8e60755) * Update pull_request_template.md (ab166a7) * Monorepo docker build fix (#46) (1dcef28) * (DOCSP-31284): Integrate prod Snooty Data API (#44) (f80ce51) * (DOCSP-31360) [UI] Conditional Streaming (#45) (34490bb) * update root package-lock (578b06b) * rename chat-ui package (d7b6dcf) * Add shared dev script to monorepo root (67811d1) * Streaming works (#25) (c173ba2) * (DOCSP-30601): Create production release system (#41) (7472c9d) * (DOCSP-31304): Implement 'all' command (#40) (f0c4cf2) * (DOCSP-31201) [UI] Updated Legalese (#39) (39c0f97) * (DOCSP-30584): ingest service CI testing and staging drone pipeline (#38) (d2e8325) * Fix + improve devcenter source tests (#37) (be0a5eb) * Fix URL building for devcenter paths (#36) (4c75175) * Add logic for ingest resumption (#35) (0f650c2) * stricter LLM to reduce halucination risk + change demo page title (#33) (cfb4628) * (DOCSP-30890) [UI] Make component mobile responsive (#29) (66080a7) * Add minor optimization for rate-limited embed requests (#34) (7547e31) * (DOCSP-31121): Add request tracking across logs (#32) (7fdc1e5) * (DOCSP-30988): Implement ingest E2E (#28) (8467989) * (DOCSP-30843): IP Address tracking and validation (#21) (7335672) * Update seed-data script (4f7a2f7) * (DOCSP-31050): Fix CI tests + run tests on PR (#30) (74fe090) * (DOCSP-30953 & DOCSP-30945): Error and edge case handling (#24) (af1b845) * Add PR template to repo (#26) (5231ef0) * Fix tests (#27) (58599ea) * more specific linter targets (18d2f5d) * Move and refactor EmbeddedContent/Store and clean up core services (#19) (43078a2) * separate format from lint (3a817b7) * remove console logs (#22) (20027ce) * Add lists to LGMarkdown (#20) (14cf986) * (DOCSP-30964): Staging build (#18) (1163ff6) * Fix renames (945337e) * Rename "chunks" to "embedded content" (#17) (a18bb8d) * Implement Data Sources (#12) (e9c5784) * Move reusable pieces to chat-core (#14) (3babf35) * (DOCSP-30577): POST `/conversations/:conversationId/messages/:messageId/rating` endpoint (#16) (c659173) * E2E (#15) (c85a2aa) * failed message handling (16acff7) * better prompt validation error handling (6408669) * UI tweaks (f803938) * (DOCSP-30921) [UI] Add prompt validation & error handling (1) (cb98ec7) * (DOCSP-30616): `POST /conversations/:conversationId/messages/` route, no streaming (#13) (8451c08) * Add publishing pipeline (42ab17e) * (DOCSP-30575): POST /conversations/ route (#11) (d609171) * (DOCSP-30619, DOCSP-30758, DOCSP-30722): Add MongoDB integration and conversation/content services (#9) (020a501) * Implement ingest store/database (#8) (5b9f8f6) * Add initial design docs (#1) (31d9069) * (DOCSP-30585): Generate search index script (#10) (458403f) * Merge branch 'demodalize' of github.com:nlarew/docs-chatbot (8d46f37) * better avatars (4c7c712) * tweak card heading (40d1dff) * loading ui (58256da) * (DOCSP-30617): OpenAI LLM service (#7) (d4af1f9) * refactor + loading states (4418f8d) * (DOCSP-30618): Create embeddings service (#6) (c54d123) * mock out Q&A API (9d80c20) * Refactor to use a single chat input (6ac351a) * Animation works except for Card ref (2bc2b41) * Rename & refactor (b5385af) * Send button for all inputs with text (9ca1873) * Move input state up (664b456) * Animation works (b8c2f42) * Use card modal WIP (27c2392) * cleanup (d96bcf0) * Fix bad imports (524f896) * (DOCSP-30576): Stub out Q&A server (#4) (899c0a3) * LGMarkdown (33596cf) * Working on LGMarkdown (897b141) * consistent models with backend (319b2ae) * UI lint pass (d8051d9) * Leafie avatar (ebd660b) * Don't require React Context for conversations (86c90b6) * fix modal style (6ae1500) * Add storybook (73a4ec3) * Modal (cac03d2) * fix banner width (088d858) * Merge pull request #3 from mongodb/DOCSP-30565 (5264da5) * Update .drone.yml (a9e3ba5) * working staging build + updated untested prod (fe16f6c) * remove uncreated env secrets (318c501) * fix broken registry path (a1e1dff) * change release (072100f) * update environment info (ff22f4f) * empty state (a66ce15) * fix value file location (ee4f06b) * fix pipeline (89f9e8b) * Trigger build (408d7dc) * Add (broken) tests (329d9e0) * Trigger build (6cae68f) * Fix context (7d213c0) * update path to docker file (f0edac3) * trigger build (e5e64c0) * more debugging (2dfb919) * Use dependency injection for PageStore/ChunkStore interfaces (89577b8) * Remove start comment (287182d) * move drone file (7453f99) * trigger build? (95579f6) * test deploy pipeline (aab4632) * add cta (db01cdb) * apply prettier (c49d368) * remove unused imports (85f6ab3) * add prettier (c1cf166) * combined props (ae9d66c) * chat state works (e29ee25) * add temp aria-labelledby prop (a0d4695) * Scaffold the ingest CLI (81a5c41) * Update ingest/README (65d2902) * chatbot-ui -> chat-ui (4340d1a) * Merge pull request #2 from mongodben/DOCSP-30486 (17662e6) * Bootstrap ingest CLI tool (0590da2) * Add chatbot-ui (997bec5) * set up QA server project structure (7cd25d5) * Delete specs directory (6b06d68) * ready repo for first push (2b250da)

* update package files (05a7c3d) * (EAI-140): Rename directories to match package names (#257) (0519375) * (EAI-49): Add static site endpoint to `mongodb-chatbot-server` + include in implementations (#255) (905c23b) * Add WIP disclaimers + sidebar update to docs (#256) (6ef3833) * (EAI-44): Ingest Prisma docs (#250) (178fcdb) * (EAI-36): WiredTiger data source (#251) (054435b) * Release mongodb-rag-core-v0.1.0 (#254) (0556fa9) * Fix broken CI (#253) (7e44368) * Release mongodb-chatbot-server v0.2.0 (#252) (7dbd308) * (EAI-28): Ingest MongoDB University transcript content (#247) (8d7bed9) * Release chat-ui v0.1.2 (c7edbdd) * Upgrade lockfile (90c2a15) * (EAI-145) InputBar adds newlines on shift+enter (#248) (b6f6d84) * (EAI-150) [UI] change "Further Reading" in chatbot response to "Related Resources" (#249) (401771e) * Create sampleMdxFile.mdx (802a614) * (EAI-138): Refactor origin handling, add custom conversation data logic, add conversations middleware (#244) (ce35576) * Deploy docs to https://mongodb.github.io/chatbot/ (#246) (af8ece8) * (EAI-18): Docs for RAG framework (#243) (ef297b2) * Release chat-ui v0.1.1 (f612ffb) * Update exports & format (d159105) * Ubiquitous dark mode (af7c8b8) * Remove unused props & update README (9341493) * Release chat-ui v0.1.0 (8a1853b) * Use a more specific env variable name (f8f4527) * Release chat-ui v0.1.0-0 (65078b7) * Remove @release-it/bumper from chat-ui (9052701) * Remove pending github token (38966a7) * Update pull_request_template.md to use new Jira project (810cb5c) * (EAI-7) [UI] Can't send existing text after reopening modal (#242) (2fbe68f) * (EAI-141): Add link to dev center blog post in README (#241) (6ab06a2) * Initial OSS Refactor (#208) (bba6322) * [Snyk] Upgrade mongodb from 5.8.1 to 5.9.1 (#238) (be8f145) * (EAI-137) [Q&A] Custom request origins (#237) (7d08d4e) * Artifactory auth with token instead of username/password (#239) (a6d1521) * (DOCSP-33030) [Q&A] Record request origin for usage tracking (#234) (eb13f4b) * (EAI-123) [UI] Replace chat-core with mongodb-rag-core (#236) (2afbdce) * (DOCSP-33938) [UI] Fix nested newline spacing (#233) (a879642) * (DOCSP-34007) [UI] Font size & spacing issues on live docs site (#235) (7b4bd3c) * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) (b8541a7) * (DOCSP-32603): Add Terraform data source (#205) (0d64743) * dev QoL scripts (3813ad7) * (DOCSP-33744): update QA server DB name for latest snapshot (#210) (5df2bf0) * (DOCSP-32482): Build out qualitative tests (#201) (0535a89) * Conversations refactor (#199) (5a60a15) * Release ingest-v0.1.2 (#204) (ed34507) * [Snyk] Upgrade urllib from 3.19.1 to 3.19.2 (#203) (743cc01) * Add Rust and Entity Framework Snooty data sources (#202) (ff15139) * [Snyk] Upgrade urllib from 3.19.0 to 3.19.1 (#200) (dea05e0) * Release chat-server v0.1.3 (ac542c8) * Use GITHUB_TOKEN for chat-ui releases (38a9e1a) * [Snyk] Upgrade urllib from 3.18.1 to 3.19.0 (#197) (dd2baaf) * Do not fail query on rephrase fail (#196) (8b7c60a) * Release chat-ui v0.0.8 (03bf7cb) * Bundle new @lg-chat components (2f1f11b) * Release chat-ui v0.0.7 (d082a63) * Unify release-it config (419422e) * Release chat-server v0.1.2 (b0b425c) * [Snyk] Upgrade urllib from 3.18.0 to 3.18.1 (#194) (2c98ca4) * [DOCSP-32950] Create a DevCenter Chatbot Component (#193) (c219d03) * (DOCSP-33445) [ANALYTICS] Scrubbed Views (#195) (8ef2bd7) * (DOCSP-33419) [ANALYTICS] Operational Views (#192) (aae8259) * Release scripts v0.1.2 (fbf2bbf) * Add whether message was deemed "Do not answer" to conversation (#191) (955b722) * (DOCSP-31104) [UI] Add tests to FE services (#190) (d3e2000) * Release ingest v0.1.1 (5b06cd2) * Update package-lock.json (a568d00) * Replace GitHub data source (#188) (747d08e) * Update env vars for analyze script (0a46085) * Add scrubbed message analyzer (#187) (9cef6b8) * (DOCSP-33291) [UI] Add tck tracking to links (#185) (acfb6a1) * Update README.md (a38cbf7) * Release 0.1.1 (8ee27c1) * Fix typo in scripts build (0534458) * Release 0.1.0 (09fa262) * Add release-it to scripts (9ea2e71) * Trigger drone run for scripts service (b1ffcc8) * Fix drone scripts (#186) (9bae8e9) * Cron scrub (#184) (d67dda9) * (DOCSP-33273) Open Source Licensing (#183) (00611bf) * De-flake SnootyProjectsInfo test (#181) (9f7250e) * (DOCSP-32930): Add pymongo metadata (#179) (6694209) * Update release-it git add config to add monorepo stuff (679e447) * Update package-lock in monorepo for release (e7308ef) * Release 0.1.1 (d60d14b) * Update release-it pre-hooks (718b96d) * Run npm audit fix (#178) (c74e1a9) * Remove `seed-content` package (#157) (1b9ac43) * Add scrub message script (#177) (ee2f864) * Remove extra padding around initial input (9dc27ad) * (DOCSP-33094): Save conversation embeddings (#175) (9c6e79d) * Ingest release 0.1.0 (#174) (4928eed) * Release chat-ui-v0.0.6 (2bf2a0e) * (DOCSP-33059) [UI] Fix suggested prompts & other LG Chat tweaks (#171) (cde1b74) * chat server v0.1.0 (#170) (ba69ad8) * new commit for build (56ed01e) * trigger ingest build (4052af9) * clean up spec + remove ip address validation from rate msg (#166) (28cf222) * UI Tweaks (#159) (53b392f) * Delete chat-server/trigger.me (#165) (5542f41) * update repo name in code (#163) (f9628bb) * Update README.md (5ad9cd0) * Launch pre releases (#162) (9b2380b) * Expand Contributing Guide and `ingest` README (#161) (30434e3) * filter out very small chunks (#160) (d952c6d) * (DOCSP-31928): Implement numbered lists (#154) (f41258d) * clarify ingest readme (115b00b) * move git install to correct docker stage (40600cb) * (DOCSP-32991, DOCSP-32992): `GitDataSource`, functionality for converting HTML to Markdown, ingest Scala, Java Reactive Streams, and C docs (#152) (7118444) * (DOCSP-32915) [UI] Streamer bugs out when switching tabs (#156) (91c288d) * (DOCSP-33090) [UI] Remove experimental badge from input on focus (#155) (a3ea236) * (DOCSP-32993, DOCSP-32906, DOCSP-32993, DOCSP-32990): Data sources for MD on Github + data ingestion of related content (#148) (7daed63) * change recommended prompt (#150) (2521f08) * add missing env example (#153) (1201a34) * [Snyk] Upgrade mongodb from 5.8.0 to 5.8.1 (#151) (966a4bf) * make preprocessor less biased toward shell (#149) (886cbe5) * (DOCSP-32616): Refactor config, add library, and log config (#143) (4fcfc58) * add default + remove unused style (0296b42) * Update chat-ui release config (3174b61) * Release chat-ui-v0.0.5 (65727f7) * Add root release:chat-ui command (4621989) * (DOCSP-32757) [UI] Update to Responsive LG Chat Components (#145) (81b70bb) * (DOCSP-32670): Basic server load testing (#136) (282c947) * [Snyk] Upgrade urllib from 3.17.2 to 3.18.0 (#147) (aefa9e0) * (DOCSP-32756): Implement headings in tables (#140) (ea416ce) * custom/synthetic/meta data source content (#127) (d687312) * [Snyk] Upgrade urllib from 3.17.1 to 3.17.2 (#144) (209135a) * (DOCSP-32673) [UI] Suppress focus to prevent modal reopen (#139) (43b8a98) * (DOCSP-32936): Increase single IP rate limits to be more line w firewall rate limits (#141) (87d8785) * (DOCSP-31863): Fix flaky test, refactor core llm service + server implementation (#142) (584a5e3) * (DOCSP-32916): Conversation service rate limiting + slow down (#138) (a88a5d5) * (DOCSP-32914): Stop unindenting context (#137) (27630c6) * (DOCSP-31285): Ingest PyMongo docs (#118) (e04c10b) * (DOCSP-32694) Dark mode: use LG palette & provider + cleanup (a95f5c5) * (DOCSP-32694) [UI] Add darkMode prop (#129) (820229c) * (DOCSP-32896) Remove mention of <CONTEXT> to reduce damage of prompt leakage (#135) (1484426) * Update internal docs (#134) (a3b2f56) * Handle Snooty Data API tombstones (#133) (a775da4) * (DOCSP-32867): Update QA DB name for sept 1 release (#131) (ce4fa3f) * (DOCSP-32868): only boost 2 results + order results by score (#130) (5230992) * (DOCSP-32856) Add a toggle for streamed vs awaited responses to the demo (#126) (94e8a20) * (DOCSP-32808): k8s production readiness (#125) (2cdf539) * Upgrade azure openai (#128) (6b85e63) * (DOCSP-32759) [UI] Use text instead of a banner for "validate answers" (#124) (302b3c7) * (DOCSP-32604) More helpful max messages error message (#123) (1be4087) * (DOCSP-32254): Include information about latest product versions + minor prompt engineering (#116) (2e40123) * (DOCSP-32657): Remove same IP address validation on conversations (#112) (731ec20) * (DOCSP-31108): Add prod index scripts (#120) (3efe895) * (DOCSP-32807): Add pm2 for running server (#122) (f659e4b) * [Snyk] Security upgrade mongodb from 5.7.0 to 5.8.0 (#121) (c985fef) * Autoscaling fix (#119) (641199a) * Source GIT_COMMIT env variable (#95) (647b00a) * Allow DOP origins in staging (#117) (38a9b66) * (DOCSP-32074): Handle dev center page metadata (#96) (e283121) * Release chat-ui-v0.0.4 (d3bad60) * (DOCSP-32371): Allow for embedded content re-ingest after changes to chunking algorithm (#114) (8b9d68e) * (DOCSP-32279): Update prompts to not mention context when don't know answer (#113) (71ca44b) * (DOCSP-32688) Stop dereferencing spec (#115) (add0dd8) * (DOCSP-31438): Horizontal pod autoscaling (#110) (3c124d4) * fix registry (d52ff7d) * Release chat-ui-v0.0.3 (f8113b7) * Add chat-ui release-it config (731e3d6) * Switch Chatbot to default export (b5db7f0) * Chat UI v0.0.2 (#111) (7e5d684) * (DOCSP-32659): Make server call its own base url if not given a full URL path (#109) (6cdd026) * Allow users to override server URL via props (#108) (708e614) * (DOCSP-32547): Update prod env to use dedicated Azure deployment (#104) (ac93735) * Prod drone fixes (#107) (4df6432) * Update QA server DB (#105) (a12e7c9) * (DOCSP-32452): Dev Center data source remove `<img>` and `<div>` tags + youtube directive (#93) (c5dac2f) * (DOCSP-32596): Set up CORS on the server (#103) (76c9530) * (DOCSP-32104): Do not serve demo site in prod environment (#101) (5f0ea11) * (DOCSP-32247): Fix preprocessed content not added to DB (#100) (57cc24e) * fix handling of openapi specs (#99) (6f9b0e4) * fix hanging promise in async recursive operation (#98) (c974058) * Drone fix (#97) (e54a272) * (DOCSP-32363) [UI] handle cases where the LLM stops mid code example (#85) (9e26c7f) * (DOCSP-32155) [UI] UX Feedback (#81) (c34b246) * (DOCSP-32243) Handle and test pages with page-level code block select (#94) (f6c3f7e) * (DOCSP-32490) [UI] Link to current commit on GitHub (#90) (64dbd0d) * (DOCSP-32242): Handle tabbed Snooty content (#92) (f87f67d) * Restore 'tags' and use 'tagsIn' only internally (#91) (035f018) * (DOCSP-32227): Improve OpenAPI spec ingestion (#88) (f369218) * (DOCSP-31343): Add system diagram and info for ingest (#86) (6e4402b) * (DOCSP-32362): Improve conversation request logging (#89) (b251e0d) * (DOCSP-32253): Add more semantically relevant product names to metadata (#83) (5869f22) * (DOCSP-32434) [UI] Add error text for non-modal input (#87) (5de9282) * (DOCSP-32206): Strip comments from rst -> md (#84) (611a50d) * (DOCSP-32194): Fix drone and k8s for ingest/chat-server staging (#80) (b61c9bc) * Define engines to remove EBADENGINE warning (#82) (f333ef5) * (DOCSP-31440, DOCSP-32075): Add tags to chunk metadata (#78) (254ce2b) * (DOCSP-32217): Change chunker to chunk based on number of tokens + add max context tokens per message (#79) (e7c1cb6) * (DOCSP-32183 & DOCSP-32226): Automate test creation based on YAML file (#77) (30eb660) * Add database index creation scripts (#75) (53f296f) * (DOCSP-31575): LLM qualitative testing framework (#74) (f222e54) * (DOCSP-31111) [UI] publish react component to npm (#73) (3baf40e) * Refactor `ConversationService` class as `makeConversationService` func (#76) (6cb9a83) * (DOCSP-32070): LLM preprocessing on user queries (#72) (6ff4219) * (DOCSP-31925): Support tables in parsed snooty MD (#67) (539d403) * Reduce ChatGPT API flakiness + faster timeout when flaky (#69) (8938599) * Update Chat UI Packaging (#71) (2e5a0fb) * (DOCSP-32066): Q&A server reduce amount of backoff on embedding (#68) (fe938b0) * (DOCSP-31639): fix timeout err in tests (#70) (741ea29) * Implement page titles and prepending frontmatter to chunks (#64) (cf56a26) * Fix JS error surfaced in the UI (#66) (5e6866a) * bundle LG Chat with component lib (c4fc213) * Modify ui config for pre-release (135d211) * (DOCSP-31154) [UI] Handle timeout & "conversation too long" errors (#65) (1943608) * (DOCSP-31791): Boost manual in the results for short queries + make app config file (#63) (6449993) * (DOCSP-31523) [UI] Replace custom components with LG Chat (#61) (3ee640b) * Fix ingest drone + err handling (#59) (8307750) * Fix code block lang (#62) (cffcbf0) * Get current branch from Snooty Data API plus allow override (#56) (6114e7c) * Clean up pages for embedding content (#60) (45ce277) * (DOCSP-31799): Parse definition lists (and other relevant content inside of `"target"` nodes) (#58) (2d7d1b3) * rename chat-ui package (#57) (dc7794f) * (DOCSP-31574): Create qualitative testing env (#55) (0a1c92a) * Add tests for data deletion (#48) (9b1417a) * Update Snooty Data Source to strip 'index' (#50) (d6b0b2f) * Add ability to override Snooty project baseUrl (#52) (fccc8ca) * Split up tests + add utils (#53) (0dedc6f) * (DOCSP-31436) [Q&A] Investigate & improve streaming rate (#54) (cc90cd0) * (DOCSP-31120) [Q&A] separate further reading reference links from LLM response (#43) (a604d83) * (DOCSP-30863) [Q&A] type check requests (#23) (5f908c0) * Streamline/nitpick/bikeshed the PR template (#51) (28b4e1b) * (DOCSP-31601): Update server base URL (#49) (1c60a0b) * (DOCSP-31573): Update prompts to address bugs (#47) (a366803) * (DOCSP-30602): Ingest service prod pipeline (#42) (8e60755) * Update pull_request_template.md (ab166a7) * Monorepo docker build fix (#46) (1dcef28) * (DOCSP-31284): Integrate prod Snooty Data API (#44) (f80ce51) * (DOCSP-31360) [UI] Conditional Streaming (#45) (34490bb) * update root package-lock (578b06b) * rename chat-ui package (d7b6dcf) * Add shared dev script to monorepo root (67811d1) * Streaming works (#25) (c173ba2) * (DOCSP-30601): Create production release system (#41) (7472c9d) * (DOCSP-31304): Implement 'all' command (#40) (f0c4cf2) * (DOCSP-31201) [UI] Updated Legalese (#39) (39c0f97) * (DOCSP-30584): ingest service CI testing and staging drone pipeline (#38) (d2e8325) * Fix + improve devcenter source tests (#37) (be0a5eb) * Fix URL building for devcenter paths (#36) (4c75175) * Add logic for ingest resumption (#35) (0f650c2) * stricter LLM to reduce halucination risk + change demo page title (#33) (cfb4628) * (DOCSP-30890) [UI] Make component mobile responsive (#29) (66080a7) * Add minor optimization for rate-limited embed requests (#34) (7547e31) * (DOCSP-31121): Add request tracking across logs (#32) (7fdc1e5) * (DOCSP-30988): Implement ingest E2E (#28) (8467989) * (DOCSP-30843): IP Address tracking and validation (#21) (7335672) * Update seed-data script (4f7a2f7) * (DOCSP-31050): Fix CI tests + run tests on PR (#30) (74fe090) * (DOCSP-30953 & DOCSP-30945): Error and edge case handling (#24) (af1b845) * Add PR template to repo (#26) (5231ef0) * Fix tests (#27) (58599ea) * more specific linter targets (18d2f5d) * Move and refactor EmbeddedContent/Store and clean up core services (#19) (43078a2) * separate format from lint (3a817b7) * remove console logs (#22) (20027ce) * Add lists to LGMarkdown (#20) (14cf986) * (DOCSP-30964): Staging build (#18) (1163ff6) * Fix renames (945337e) * Rename "chunks" to "embedded content" (#17) (a18bb8d) * Implement Data Sources (#12) (e9c5784) * Move reusable pieces to chat-core (#14) (3babf35) * (DOCSP-30577): POST `/conversations/:conversationId/messages/:messageId/rating` endpoint (#16) (c659173) * E2E (#15) (c85a2aa) * failed message handling (16acff7) * better prompt validation error handling (6408669) * UI tweaks (f803938) * (DOCSP-30921) [UI] Add prompt validation & error handling (1) (cb98ec7) * (DOCSP-30616): `POST /conversations/:conversationId/messages/` route, no streaming (#13) (8451c08) * Add publishing pipeline (42ab17e) * (DOCSP-30575): POST /conversations/ route (#11) (d609171) * (DOCSP-30619, DOCSP-30758, DOCSP-30722): Add MongoDB integration and conversation/content services (#9) (020a501) * Implement ingest store/database (#8) (5b9f8f6) * Add initial design docs (#1) (31d9069) * (DOCSP-30585): Generate search index script (#10) (458403f) * Merge branch 'demodalize' of github.com:nlarew/docs-chatbot (8d46f37) * better avatars (4c7c712) * tweak card heading (40d1dff) * loading ui (58256da) * (DOCSP-30617): OpenAI LLM service (#7) (d4af1f9) * refactor + loading states (4418f8d) * (DOCSP-30618): Create embeddings service (#6) (c54d123) * mock out Q&A API (9d80c20) * Refactor to use a single chat input (6ac351a) * Animation works except for Card ref (2bc2b41) * Rename & refactor (b5385af) * Send button for all inputs with text (9ca1873) * Move input state up (664b456) * Animation works (b8c2f42) * Use card modal WIP (27c2392) * cleanup (d96bcf0) * Fix bad imports (524f896) * (DOCSP-30576): Stub out Q&A server (#4) (899c0a3) * LGMarkdown (33596cf) * Working on LGMarkdown (897b141) * consistent models with backend (319b2ae) * UI lint pass (d8051d9) * Leafie avatar (ebd660b) * Don't require React Context for conversations (86c90b6) * fix modal style (6ae1500) * Add storybook (73a4ec3) * Modal (cac03d2) * fix banner width (088d858) * Merge pull request #3 from mongodb/DOCSP-30565 (5264da5) * Update .drone.yml (a9e3ba5) * working staging build + updated untested prod (fe16f6c) * remove uncreated env secrets (318c501) * fix broken registry path (a1e1dff) * change release (072100f) * update environment info (ff22f4f) * empty state (a66ce15) * fix value file location (ee4f06b) * fix pipeline (89f9e8b) * Trigger build (408d7dc) * Add (broken) tests (329d9e0) * Trigger build (6cae68f) * Fix context (7d213c0) * update path to docker file (f0edac3) * trigger build (e5e64c0) * more debugging (2dfb919) * Use dependency injection for PageStore/ChunkStore interfaces (89577b8) * Remove start comment (287182d) * move drone file (7453f99) * trigger build? (95579f6) * test deploy pipeline (aab4632) * add cta (db01cdb) * apply prettier (c49d368) * remove unused imports (85f6ab3) * add prettier (c1cf166) * combined props (ae9d66c) * chat state works (e29ee25) * add temp aria-labelledby prop (a0d4695) * Scaffold the ingest CLI (81a5c41) * Update ingest/README (65d2902) * chatbot-ui -> chat-ui (4340d1a) * Merge pull request #2 from mongodben/DOCSP-30486 (17662e6) * Bootstrap ingest CLI tool (0590da2) * Add chatbot-ui (997bec5) * set up QA server project structure (7cd25d5) * Delete specs directory (6b06d68) * ready repo for first push (2b250da)

* update package files * Release mongodb-rag-ingest v0.1.7 * update package files (05a7c3d) * (EAI-140): Rename directories to match package names (#257) (0519375) * (EAI-49): Add static site endpoint to `mongodb-chatbot-server` + include in implementations (#255) (905c23b) * Add WIP disclaimers + sidebar update to docs (#256) (6ef3833) * (EAI-44): Ingest Prisma docs (#250) (178fcdb) * (EAI-36): WiredTiger data source (#251) (054435b) * Release mongodb-rag-core-v0.1.0 (#254) (0556fa9) * Fix broken CI (#253) (7e44368) * Release mongodb-chatbot-server v0.2.0 (#252) (7dbd308) * (EAI-28): Ingest MongoDB University transcript content (#247) (8d7bed9) * Release chat-ui v0.1.2 (c7edbdd) * Upgrade lockfile (90c2a15) * (EAI-145) InputBar adds newlines on shift+enter (#248) (b6f6d84) * (EAI-150) [UI] change "Further Reading" in chatbot response to "Related Resources" (#249) (401771e) * Create sampleMdxFile.mdx (802a614) * (EAI-138): Refactor origin handling, add custom conversation data logic, add conversations middleware (#244) (ce35576) * Deploy docs to https://mongodb.github.io/chatbot/ (#246) (af8ece8) * (EAI-18): Docs for RAG framework (#243) (ef297b2) * Release chat-ui v0.1.1 (f612ffb) * Update exports & format (d159105) * Ubiquitous dark mode (af7c8b8) * Remove unused props & update README (9341493) * Release chat-ui v0.1.0 (8a1853b) * Use a more specific env variable name (f8f4527) * Release chat-ui v0.1.0-0 (65078b7) * Remove @release-it/bumper from chat-ui (9052701) * Remove pending github token (38966a7) * Update pull_request_template.md to use new Jira project (810cb5c) * (EAI-7) [UI] Can't send existing text after reopening modal (#242) (2fbe68f) * (EAI-141): Add link to dev center blog post in README (#241) (6ab06a2) * Initial OSS Refactor (#208) (bba6322) * [Snyk] Upgrade mongodb from 5.8.1 to 5.9.1 (#238) (be8f145) * (EAI-137) [Q&A] Custom request origins (#237) (7d08d4e) * Artifactory auth with token instead of username/password (#239) (a6d1521) * (DOCSP-33030) [Q&A] Record request origin for usage tracking (#234) (eb13f4b) * (EAI-123) [UI] Replace chat-core with mongodb-rag-core (#236) (2afbdce) * (DOCSP-33938) [UI] Fix nested newline spacing (#233) (a879642) * (DOCSP-34007) [UI] Font size & spacing issues on live docs site (#235) (7b4bd3c) * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) (b8541a7) * (DOCSP-32603): Add Terraform data source (#205) (0d64743) * dev QoL scripts (3813ad7) * (DOCSP-33744): update QA server DB name for latest snapshot (#210) (5df2bf0) * (DOCSP-32482): Build out qualitative tests (#201) (0535a89) * Conversations refactor (#199) (5a60a15) * Release ingest-v0.1.2 (#204) (ed34507) * [Snyk] Upgrade urllib from 3.19.1 to 3.19.2 (#203) (743cc01) * Add Rust and Entity Framework Snooty data sources (#202) (ff15139) * [Snyk] Upgrade urllib from 3.19.0 to 3.19.1 (#200) (dea05e0) * Release chat-server v0.1.3 (ac542c8) * Use GITHUB_TOKEN for chat-ui releases (38a9e1a) * [Snyk] Upgrade urllib from 3.18.1 to 3.19.0 (#197) (dd2baaf) * Do not fail query on rephrase fail (#196) (8b7c60a) * Release chat-ui v0.0.8 (03bf7cb) * Bundle new @lg-chat components (2f1f11b) * Release chat-ui v0.0.7 (d082a63) * Unify release-it config (419422e) * Release chat-server v0.1.2 (b0b425c) * [Snyk] Upgrade urllib from 3.18.0 to 3.18.1 (#194) (2c98ca4) * [DOCSP-32950] Create a DevCenter Chatbot Component (#193) (c219d03) * (DOCSP-33445) [ANALYTICS] Scrubbed Views (#195) (8ef2bd7) * (DOCSP-33419) [ANALYTICS] Operational Views (#192) (aae8259) * Release scripts v0.1.2 (fbf2bbf) * Add whether message was deemed "Do not answer" to conversation (#191) (955b722) * (DOCSP-31104) [UI] Add tests to FE services (#190) (d3e2000) * Release ingest v0.1.1 (5b06cd2) * Update package-lock.json (a568d00) * Replace GitHub data source (#188) (747d08e) * Update env vars for analyze script (0a46085) * Add scrubbed message analyzer (#187) (9cef6b8) * (DOCSP-33291) [UI] Add tck tracking to links (#185) (acfb6a1) * Update README.md (a38cbf7) * Release 0.1.1 (8ee27c1) * Fix typo in scripts build (0534458) * Release 0.1.0 (09fa262) * Add release-it to scripts (9ea2e71) * Trigger drone run for scripts service (b1ffcc8) * Fix drone scripts (#186) (9bae8e9) * Cron scrub (#184) (d67dda9) * (DOCSP-33273) Open Source Licensing (#183) (00611bf) * De-flake SnootyProjectsInfo test (#181) (9f7250e) * (DOCSP-32930): Add pymongo metadata (#179) (6694209) * Update release-it git add config to add monorepo stuff (679e447) * Update package-lock in monorepo for release (e7308ef) * Release 0.1.1 (d60d14b) * Update release-it pre-hooks (718b96d) * Run npm audit fix (#178) (c74e1a9) * Remove `seed-content` package (#157) (1b9ac43) * Add scrub message script (#177) (ee2f864) * Remove extra padding around initial input (9dc27ad) * (DOCSP-33094): Save conversation embeddings (#175) (9c6e79d) * Ingest release 0.1.0 (#174) (4928eed) * Release chat-ui-v0.0.6 (2bf2a0e) * (DOCSP-33059) [UI] Fix suggested prompts & other LG Chat tweaks (#171) (cde1b74) * chat server v0.1.0 (#170) (ba69ad8) * new commit for build (56ed01e) * trigger ingest build (4052af9) * clean up spec + remove ip address validation from rate msg (#166) (28cf222) * UI Tweaks (#159) (53b392f) * Delete chat-server/trigger.me (#165) (5542f41) * update repo name in code (#163) (f9628bb) * Update README.md (5ad9cd0) * Launch pre releases (#162) (9b2380b) * Expand Contributing Guide and `ingest` README (#161) (30434e3) * filter out very small chunks (#160) (d952c6d) * (DOCSP-31928): Implement numbered lists (#154) (f41258d) * clarify ingest readme (115b00b) * move git install to correct docker stage (40600cb) * (DOCSP-32991, DOCSP-32992): `GitDataSource`, functionality for converting HTML to Markdown, ingest Scala, Java Reactive Streams, and C docs (#152) (7118444) * (DOCSP-32915) [UI] Streamer bugs out when switching tabs (#156) (91c288d) * (DOCSP-33090) [UI] Remove experimental badge from input on focus (#155) (a3ea236) * (DOCSP-32993, DOCSP-32906, DOCSP-32993, DOCSP-32990): Data sources for MD on Github + data ingestion of related content (#148) (7daed63) * change recommended prompt (#150) (2521f08) * add missing env example (#153) (1201a34) * [Snyk] Upgrade mongodb from 5.8.0 to 5.8.1 (#151) (966a4bf) * make preprocessor less biased toward shell (#149) (886cbe5) * (DOCSP-32616): Refactor config, add library, and log config (#143) (4fcfc58) * add default + remove unused style (0296b42) * Update chat-ui release config (3174b61) * Release chat-ui-v0.0.5 (65727f7) * Add root release:chat-ui command (4621989) * (DOCSP-32757) [UI] Update to Responsive LG Chat Components (#145) (81b70bb) * (DOCSP-32670): Basic server load testing (#136) (282c947) * [Snyk] Upgrade urllib from 3.17.2 to 3.18.0 (#147) (aefa9e0) * (DOCSP-32756): Implement headings in tables (#140) (ea416ce) * custom/synthetic/meta data source content (#127) (d687312) * [Snyk] Upgrade urllib from 3.17.1 to 3.17.2 (#144) (209135a) * (DOCSP-32673) [UI] Suppress focus to prevent modal reopen (#139) (43b8a98) * (DOCSP-32936): Increase single IP rate limits to be more line w firewall rate limits (#141) (87d8785) * (DOCSP-31863): Fix flaky test, refactor core llm service + server implementation (#142) (584a5e3) * (DOCSP-32916): Conversation service rate limiting + slow down (#138) (a88a5d5) * (DOCSP-32914): Stop unindenting context (#137) (27630c6) * (DOCSP-31285): Ingest PyMongo docs (#118) (e04c10b) * (DOCSP-32694) Dark mode: use LG palette & provider + cleanup (a95f5c5) * (DOCSP-32694) [UI] Add darkMode prop (#129) (820229c) * (DOCSP-32896) Remove mention of <CONTEXT> to reduce damage of prompt leakage (#135) (1484426) * Update internal docs (#134) (a3b2f56) * Handle Snooty Data API tombstones (#133) (a775da4) * (DOCSP-32867): Update QA DB name for sept 1 release (#131) (ce4fa3f) * (DOCSP-32868): only boost 2 results + order results by score (#130) (5230992) * (DOCSP-32856) Add a toggle for streamed vs awaited responses to the demo (#126) (94e8a20) * (DOCSP-32808): k8s production readiness (#125) (2cdf539) * Upgrade azure openai (#128) (6b85e63) * (DOCSP-32759) [UI] Use text instead of a banner for "validate answers" (#124) (302b3c7) * (DOCSP-32604) More helpful max messages error message (#123) (1be4087) * (DOCSP-32254): Include information about latest product versions + minor prompt engineering (#116) (2e40123) * (DOCSP-32657): Remove same IP address validation on conversations (#112) (731ec20) * (DOCSP-31108): Add prod index scripts (#120) (3efe895) * (DOCSP-32807): Add pm2 for running server (#122) (f659e4b) * [Snyk] Security upgrade mongodb from 5.7.0 to 5.8.0 (#121) (c985fef) * Autoscaling fix (#119) (641199a) * Source GIT_COMMIT env variable (#95) (647b00a) * Allow DOP origins in staging (#117) (38a9b66) * (DOCSP-32074): Handle dev center page metadata (#96) (e283121) * Release chat-ui-v0.0.4 (d3bad60) * (DOCSP-32371): Allow for embedded content re-ingest after changes to chunking algorithm (#114) (8b9d68e) * (DOCSP-32279): Update prompts to not mention context when don't know answer (#113) (71ca44b) * (DOCSP-32688) Stop dereferencing spec (#115) (add0dd8) * (DOCSP-31438): Horizontal pod autoscaling (#110) (3c124d4) * fix registry (d52ff7d) * Release chat-ui-v0.0.3 (f8113b7) * Add chat-ui release-it config (731e3d6) * Switch Chatbot to default export (b5db7f0) * Chat UI v0.0.2 (#111) (7e5d684) * (DOCSP-32659): Make server call its own base url if not given a full URL path (#109) (6cdd026) * Allow users to override server URL via props (#108) (708e614) * (DOCSP-32547): Update prod env to use dedicated Azure deployment (#104) (ac93735) * Prod drone fixes (#107) (4df6432) * Update QA server DB (#105) (a12e7c9) * (DOCSP-32452): Dev Center data source remove `<img>` and `<div>` tags + youtube directive (#93) (c5dac2f) * (DOCSP-32596): Set up CORS on the server (#103) (76c9530) * (DOCSP-32104): Do not serve demo site in prod environment (#101) (5f0ea11) * (DOCSP-32247): Fix preprocessed content not added to DB (#100) (57cc24e) * fix handling of openapi specs (#99) (6f9b0e4) * fix hanging promise in async recursive operation (#98) (c974058) * Drone fix (#97) (e54a272) * (DOCSP-32363) [UI] handle cases where the LLM stops mid code example (#85) (9e26c7f) * (DOCSP-32155) [UI] UX Feedback (#81) (c34b246) * (DOCSP-32243) Handle and test pages with page-level code block select (#94) (f6c3f7e) * (DOCSP-32490) [UI] Link to current commit on GitHub (#90) (64dbd0d) * (DOCSP-32242): Handle tabbed Snooty content (#92) (f87f67d) * Restore 'tags' and use 'tagsIn' only internally (#91) (035f018) * (DOCSP-32227): Improve OpenAPI spec ingestion (#88) (f369218) * (DOCSP-31343): Add system diagram and info for ingest (#86) (6e4402b) * (DOCSP-32362): Improve conversation request logging (#89) (b251e0d) * (DOCSP-32253): Add more semantically relevant product names to metadata (#83) (5869f22) * (DOCSP-32434) [UI] Add error text for non-modal input (#87) (5de9282) * (DOCSP-32206): Strip comments from rst -> md (#84) (611a50d) * (DOCSP-32194): Fix drone and k8s for ingest/chat-server staging (#80) (b61c9bc) * Define engines to remove EBADENGINE warning (#82) (f333ef5) * (DOCSP-31440, DOCSP-32075): Add tags to chunk metadata (#78) (254ce2b) * (DOCSP-32217): Change chunker to chunk based on number of tokens + add max context tokens per message (#79) (e7c1cb6) * (DOCSP-32183 & DOCSP-32226): Automate test creation based on YAML file (#77) (30eb660) * Add database index creation scripts (#75) (53f296f) * (DOCSP-31575): LLM qualitative testing framework (#74) (f222e54) * (DOCSP-31111) [UI] publish react component to npm (#73) (3baf40e) * Refactor `ConversationService` class as `makeConversationService` func (#76) (6cb9a83) * (DOCSP-32070): LLM preprocessing on user queries (#72) (6ff4219) * (DOCSP-31925): Support tables in parsed snooty MD (#67) (539d403) * Reduce ChatGPT API flakiness + faster timeout when flaky (#69) (8938599) * Update Chat UI Packaging (#71) (2e5a0fb) * (DOCSP-32066): Q&A server reduce amount of backoff on embedding (#68) (fe938b0) * (DOCSP-31639): fix timeout err in tests (#70) (741ea29) * Implement page titles and prepending frontmatter to chunks (#64) (cf56a26) * Fix JS error surfaced in the UI (#66) (5e6866a) * bundle LG Chat with component lib (c4fc213) * Modify ui config for pre-release (135d211) * (DOCSP-31154) [UI] Handle timeout & "conversation too long" errors (#65) (1943608) * (DOCSP-31791): Boost manual in the results for short queries + make app config file (#63) (6449993) * (DOCSP-31523) [UI] Replace custom components with LG Chat (#61) (3ee640b) * Fix ingest drone + err handling (#59) (8307750) * Fix code block lang (#62) (cffcbf0) * Get current branch from Snooty Data API plus allow override (#56) (6114e7c) * Clean up pages for embedding content (#60) (45ce277) * (DOCSP-31799): Parse definition lists (and other relevant content inside of `"target"` nodes) (#58) (2d7d1b3) * rename chat-ui package (#57) (dc7794f) * (DOCSP-31574): Create qualitative testing env (#55) (0a1c92a) * Add tests for data deletion (#48) (9b1417a) * Update Snooty Data Source to strip 'index' (#50) (d6b0b2f) * Add ability to override Snooty project baseUrl (#52) (fccc8ca) * Split up tests + add utils (#53) (0dedc6f) * (DOCSP-31436) [Q&A] Investigate & improve streaming rate (#54) (cc90cd0) * (DOCSP-31120) [Q&A] separate further reading reference links from LLM response (#43) (a604d83) * (DOCSP-30863) [Q&A] type check requests (#23) (5f908c0) * Streamline/nitpick/bikeshed the PR template (#51) (28b4e1b) * (DOCSP-31601): Update server base URL (#49) (1c60a0b) * (DOCSP-31573): Update prompts to address bugs (#47) (a366803) * (DOCSP-30602): Ingest service prod pipeline (#42) (8e60755) * Update pull_request_template.md (ab166a7) * Monorepo docker build fix (#46) (1dcef28) * (DOCSP-31284): Integrate prod Snooty Data API (#44) (f80ce51) * (DOCSP-31360) [UI] Conditional Streaming (#45) (34490bb) * update root package-lock (578b06b) * rename chat-ui package (d7b6dcf) * Add shared dev script to monorepo root (67811d1) * Streaming works (#25) (c173ba2) * (DOCSP-30601): Create production release system (#41) (7472c9d) * (DOCSP-31304): Implement 'all' command (#40) (f0c4cf2) * (DOCSP-31201) [UI] Updated Legalese (#39) (39c0f97) * (DOCSP-30584): ingest service CI testing and staging drone pipeline (#38) (d2e8325) * Fix + improve devcenter source tests (#37) (be0a5eb) * Fix URL building for devcenter paths (#36) (4c75175) * Add logic for ingest resumption (#35) (0f650c2) * stricter LLM to reduce halucination risk + change demo page title (#33) (cfb4628) * (DOCSP-30890) [UI] Make component mobile responsive (#29) (66080a7) * Add minor optimization for rate-limited embed requests (#34) (7547e31) * (DOCSP-31121): Add request tracking across logs (#32) (7fdc1e5) * (DOCSP-30988): Implement ingest E2E (#28) (8467989) * (DOCSP-30843): IP Address tracking and validation (#21) (7335672) * Update seed-data script (4f7a2f7) * (DOCSP-31050): Fix CI tests + run tests on PR (#30) (74fe090) * (DOCSP-30953 & DOCSP-30945): Error and edge case handling (#24) (af1b845) * Add PR template to repo (#26) (5231ef0) * Fix tests (#27) (58599ea) * more specific linter targets (18d2f5d) * Move and refactor EmbeddedContent/Store and clean up core services (#19) (43078a2) * separate format from lint (3a817b7) * remove console logs (#22) (20027ce) * Add lists to LGMarkdown (#20) (14cf986) * (DOCSP-30964): Staging build (#18) (1163ff6) * Fix renames (945337e) * Rename "chunks" to "embedded content" (#17) (a18bb8d) * Implement Data Sources (#12) (e9c5784) * Move reusable pieces to chat-core (#14) (3babf35) * (DOCSP-30577): POST `/conversations/:conversationId/messages/:messageId/rating` endpoint (#16) (c659173) * E2E (#15) (c85a2aa) * failed message handling (16acff7) * better prompt validation error handling (6408669) * UI tweaks (f803938) * (DOCSP-30921) [UI] Add prompt validation & error handling (1) (cb98ec7) * (DOCSP-30616): `POST /conversations/:conversationId/messages/` route, no streaming (#13) (8451c08) * Add publishing pipeline (42ab17e) * (DOCSP-30575): POST /conversations/ route (#11) (d609171) * (DOCSP-30619, DOCSP-30758, DOCSP-30722): Add MongoDB integration and conversation/content services (#9) (020a501) * Implement ingest store/database (#8) (5b9f8f6) * Add initial design docs (#1) (31d9069) * (DOCSP-30585): Generate search index script (#10) (458403f) * Merge branch 'demodalize' of github.com:nlarew/docs-chatbot (8d46f37) * better avatars (4c7c712) * tweak card heading (40d1dff) * loading ui (58256da) * (DOCSP-30617): OpenAI LLM service (#7) (d4af1f9) * refactor + loading states (4418f8d) * (DOCSP-30618): Create embeddings service (#6) (c54d123) * mock out Q&A API (9d80c20) * Refactor to use a single chat input (6ac351a) * Animation works except for Card ref (2bc2b41) * Rename & refactor (b5385af) * Send button for all inputs with text (9ca1873) * Move input state up (664b456) * Animation works (b8c2f42) * Use card modal WIP (27c2392) * cleanup (d96bcf0) * Fix bad imports (524f896) * (DOCSP-30576): Stub out Q&A server (#4) (899c0a3) * LGMarkdown (33596cf) * Working on LGMarkdown (897b141) * consistent models with backend (319b2ae) * UI lint pass (d8051d9) * Leafie avatar (ebd660b) * Don't require React Context for conversations (86c90b6) * fix modal style (6ae1500) * Add storybook (73a4ec3) * Modal (cac03d2) * fix banner width (088d858) * Merge pull request #3 from mongodb/DOCSP-30565 (5264da5) * Update .drone.yml (a9e3ba5) * working staging build + updated untested prod (fe16f6c) * remove uncreated env secrets (318c501) * fix broken registry path (a1e1dff) * change release (072100f) * update environment info (ff22f4f) * empty state (a66ce15) * fix value file location (ee4f06b) * fix pipeline (89f9e8b) * Trigger build (408d7dc) * Add (broken) tests (329d9e0) * Trigger build (6cae68f) * Fix context (7d213c0) * update path to docker file (f0edac3) * trigger build (e5e64c0) * more debugging (2dfb919) * Use dependency injection for PageStore/ChunkStore interfaces (89577b8) * Remove start comment (287182d) * move drone file (7453f99) * trigger build? (95579f6) * test deploy pipeline (aab4632) * add cta (db01cdb) * apply prettier (c49d368) * remove unused imports (85f6ab3) * add prettier (c1cf166) * combined props (ae9d66c) * chat state works (e29ee25) * add temp aria-labelledby prop (a0d4695) * Scaffold the ingest CLI (81a5c41) * Update ingest/README (65d2902) * chatbot-ui -> chat-ui (4340d1a) * Merge pull request #2 from mongodben/DOCSP-30486 (17662e6) * Bootstrap ingest CLI tool (0590da2) * Add chatbot-ui (997bec5) * set up QA server project structure (7cd25d5) * Delete specs directory (6b06d68) * ready repo for first push (2b250da)

* (EAI-140): Rename directories to match package names (#257) (0519375) * (EAI-49): Add static site endpoint to `mongodb-chatbot-server` + include in implementations (#255) (905c23b) * Add WIP disclaimers + sidebar update to docs (#256) (6ef3833) * (EAI-44): Ingest Prisma docs (#250) (178fcdb) * (EAI-36): WiredTiger data source (#251) (054435b) * Release mongodb-rag-core-v0.1.0 (#254) (0556fa9) * Fix broken CI (#253) (7e44368) * Release mongodb-chatbot-server v0.2.0 (#252) (7dbd308) * (EAI-28): Ingest MongoDB University transcript content (#247) (8d7bed9) * Release chat-ui v0.1.2 (c7edbdd) * Upgrade lockfile (90c2a15) * (EAI-145) InputBar adds newlines on shift+enter (#248) (b6f6d84) * (EAI-150) [UI] change "Further Reading" in chatbot response to "Related Resources" (#249) (401771e) * Create sampleMdxFile.mdx (802a614) * (EAI-138): Refactor origin handling, add custom conversation data logic, add conversations middleware (#244) (ce35576) * Deploy docs to https://mongodb.github.io/chatbot/ (#246) (af8ece8) * (EAI-18): Docs for RAG framework (#243) (ef297b2) * Release chat-ui v0.1.1 (f612ffb) * Update exports & format (d159105) * Ubiquitous dark mode (af7c8b8) * Remove unused props & update README (9341493) * Release chat-ui v0.1.0 (8a1853b) * Use a more specific env variable name (f8f4527) * Release chat-ui v0.1.0-0 (65078b7) * Remove @release-it/bumper from chat-ui (9052701) * Remove pending github token (38966a7) * Update pull_request_template.md to use new Jira project (810cb5c) * (EAI-7) [UI] Can't send existing text after reopening modal (#242) (2fbe68f) * (EAI-141): Add link to dev center blog post in README (#241) (6ab06a2) * Initial OSS Refactor (#208) (bba6322) * [Snyk] Upgrade mongodb from 5.8.1 to 5.9.1 (#238) (be8f145) * (EAI-137) [Q&A] Custom request origins (#237) (7d08d4e) * Artifactory auth with token instead of username/password (#239) (a6d1521) * (DOCSP-33030) [Q&A] Record request origin for usage tracking (#234) (eb13f4b) * (EAI-123) [UI] Replace chat-core with mongodb-rag-core (#236) (2afbdce) * (DOCSP-33938) [UI] Fix nested newline spacing (#233) (a879642) * (DOCSP-34007) [UI] Font size & spacing issues on live docs site (#235) (7b4bd3c) * (DOCSP-33348) [UI] Abstract chatbot component for multiple sites (#221) (b8541a7) * (DOCSP-32603): Add Terraform data source (#205) (0d64743) * dev QoL scripts (3813ad7) * (DOCSP-33744): update QA server DB name for latest snapshot (#210) (5df2bf0) * (DOCSP-32482): Build out qualitative tests (#201) (0535a89) * Conversations refactor (#199) (5a60a15) * Release ingest-v0.1.2 (#204) (ed34507) * [Snyk] Upgrade urllib from 3.19.1 to 3.19.2 (#203) (743cc01) * Add Rust and Entity Framework Snooty data sources (#202) (ff15139) * [Snyk] Upgrade urllib from 3.19.0 to 3.19.1 (#200) (dea05e0) * Release chat-server v0.1.3 (ac542c8) * Use GITHUB_TOKEN for chat-ui releases (38a9e1a) * [Snyk] Upgrade urllib from 3.18.1 to 3.19.0 (#197) (dd2baaf) * Do not fail query on rephrase fail (#196) (8b7c60a) * Release chat-ui v0.0.8 (03bf7cb) * Bundle new @lg-chat components (2f1f11b) * Release chat-ui v0.0.7 (d082a63) * Unify release-it config (419422e) * Release chat-server v0.1.2 (b0b425c) * [Snyk] Upgrade urllib from 3.18.0 to 3.18.1 (#194) (2c98ca4) * [DOCSP-32950] Create a DevCenter Chatbot Component (#193) (c219d03) * (DOCSP-33445) [ANALYTICS] Scrubbed Views (#195) (8ef2bd7) * (DOCSP-33419) [ANALYTICS] Operational Views (#192) (aae8259) * Release scripts v0.1.2 (fbf2bbf) * Add whether message was deemed "Do not answer" to conversation (#191) (955b722) * (DOCSP-31104) [UI] Add tests to FE services (#190) (d3e2000) * Release ingest v0.1.1 (5b06cd2) * Update package-lock.json (a568d00) * Replace GitHub data source (#188) (747d08e) * Update env vars for analyze script (0a46085) * Add scrubbed message analyzer (#187) (9cef6b8) * (DOCSP-33291) [UI] Add tck tracking to links (#185) (acfb6a1) * Update README.md (a38cbf7) * Release 0.1.1 (8ee27c1) * Fix typo in scripts build (0534458) * Release 0.1.0 (09fa262) * Add release-it to scripts (9ea2e71) * Trigger drone run for scripts service (b1ffcc8) * Fix drone scripts (#186) (9bae8e9) * Cron scrub (#184) (d67dda9) * (DOCSP-33273) Open Source Licensing (#183) (00611bf) * De-flake SnootyProjectsInfo test (#181) (9f7250e) * (DOCSP-32930): Add pymongo metadata (#179) (6694209) * Update release-it git add config to add monorepo stuff (679e447) * Update package-lock in monorepo for release (e7308ef) * Release 0.1.1 (d60d14b) * Update release-it pre-hooks (718b96d) * Run npm audit fix (#178) (c74e1a9) * Remove `seed-content` package (#157) (1b9ac43) * Add scrub message script (#177) (ee2f864) * Remove extra padding around initial input (9dc27ad) * (DOCSP-33094): Save conversation embeddings (#175) (9c6e79d) * Ingest release 0.1.0 (#174) (4928eed) * Release chat-ui-v0.0.6 (2bf2a0e) * (DOCSP-33059) [UI] Fix suggested prompts & other LG Chat tweaks (#171) (cde1b74) * chat server v0.1.0 (#170) (ba69ad8) * new commit for build (56ed01e) * trigger ingest build (4052af9) * clean up spec + remove ip address validation from rate msg (#166) (28cf222) * UI Tweaks (#159) (53b392f) * Delete chat-server/trigger.me (#165) (5542f41) * update repo name in code (#163) (f9628bb) * Update README.md (5ad9cd0) * Launch pre releases (#162) (9b2380b) * Expand Contributing Guide and `ingest` README (#161) (30434e3) * filter out very small chunks (#160) (d952c6d) * (DOCSP-31928): Implement numbered lists (#154) (f41258d) * clarify ingest readme (115b00b) * move git install to correct docker stage (40600cb) * (DOCSP-32991, DOCSP-32992): `GitDataSource`, functionality for converting HTML to Markdown, ingest Scala, Java Reactive Streams, and C docs (#152) (7118444) * (DOCSP-32915) [UI] Streamer bugs out when switching tabs (#156) (91c288d) * (DOCSP-33090) [UI] Remove experimental badge from input on focus (#155) (a3ea236) * (DOCSP-32993, DOCSP-32906, DOCSP-32993, DOCSP-32990): Data sources for MD on Github + data ingestion of related content (#148) (7daed63) * change recommended prompt (#150) (2521f08) * add missing env example (#153) (1201a34) * [Snyk] Upgrade mongodb from 5.8.0 to 5.8.1 (#151) (966a4bf) * make preprocessor less biased toward shell (#149) (886cbe5) * (DOCSP-32616): Refactor config, add library, and log config (#143) (4fcfc58) * add default + remove unused style (0296b42) * Update chat-ui release config (3174b61) * Release chat-ui-v0.0.5 (65727f7) * Add root release:chat-ui command (4621989) * (DOCSP-32757) [UI] Update to Responsive LG Chat Components (#145) (81b70bb) * (DOCSP-32670): Basic server load testing (#136) (282c947) * [Snyk] Upgrade urllib from 3.17.2 to 3.18.0 (#147) (aefa9e0) * (DOCSP-32756): Implement headings in tables (#140) (ea416ce) * custom/synthetic/meta data source content (#127) (d687312) * [Snyk] Upgrade urllib from 3.17.1 to 3.17.2 (#144) (209135a) * (DOCSP-32673) [UI] Suppress focus to prevent modal reopen (#139) (43b8a98) * (DOCSP-32936): Increase single IP rate limits to be more line w firewall rate limits (#141) (87d8785) * (DOCSP-31863): Fix flaky test, refactor core llm service + server implementation (#142) (584a5e3) * (DOCSP-32916): Conversation service rate limiting + slow down (#138) (a88a5d5) * (DOCSP-32914): Stop unindenting context (#137) (27630c6) * (DOCSP-31285): Ingest PyMongo docs (#118) (e04c10b) * (DOCSP-32694) Dark mode: use LG palette & provider + cleanup (a95f5c5) * (DOCSP-32694) [UI] Add darkMode prop (#129) (820229c) * (DOCSP-32896) Remove mention of <CONTEXT> to reduce damage of prompt leakage (#135) (1484426) * Update internal docs (#134) (a3b2f56) * Handle Snooty Data API tombstones (#133) (a775da4) * (DOCSP-32867): Update QA DB name for sept 1 release (#131) (ce4fa3f) * (DOCSP-32868): only boost 2 results + order results by score (#130) (5230992) * (DOCSP-32856) Add a toggle for streamed vs awaited responses to the demo (#126) (94e8a20) * (DOCSP-32808): k8s production readiness (#125) (2cdf539) * Upgrade azure openai (#128) (6b85e63) * (DOCSP-32759) [UI] Use text instead of a banner for "validate answers" (#124) (302b3c7) * (DOCSP-32604) More helpful max messages error message (#123) (1be4087) * (DOCSP-32254): Include information about latest product versions + minor prompt engineering (#116) (2e40123) * (DOCSP-32657): Remove same IP address validation on conversations (#112) (731ec20) * (DOCSP-31108): Add prod index scripts (#120) (3efe895) * (DOCSP-32807): Add pm2 for running server (#122) (f659e4b) * [Snyk] Security upgrade mongodb from 5.7.0 to 5.8.0 (#121) (c985fef) * Autoscaling fix (#119) (641199a) * Source GIT_COMMIT env variable (#95) (647b00a) * Allow DOP origins in staging (#117) (38a9b66) * (DOCSP-32074): Handle dev center page metadata (#96) (e283121) * Release chat-ui-v0.0.4 (d3bad60) * (DOCSP-32371): Allow for embedded content re-ingest after changes to chunking algorithm (#114) (8b9d68e) * (DOCSP-32279): Update prompts to not mention context when don't know answer (#113) (71ca44b) * (DOCSP-32688) Stop dereferencing spec (#115) (add0dd8) * (DOCSP-31438): Horizontal pod autoscaling (#110) (3c124d4) * fix registry (d52ff7d) * Release chat-ui-v0.0.3 (f8113b7) * Add chat-ui release-it config (731e3d6) * Switch Chatbot to default export (b5db7f0) * Chat UI v0.0.2 (#111) (7e5d684) * (DOCSP-32659): Make server call its own base url if not given a full URL path (#109) (6cdd026) * Allow users to override server URL via props (#108) (708e614) * (DOCSP-32547): Update prod env to use dedicated Azure deployment (#104) (ac93735) * Prod drone fixes (#107) (4df6432) * Update QA server DB (#105) (a12e7c9) * (DOCSP-32452): Dev Center data source remove `<img>` and `<div>` tags + youtube directive (#93) (c5dac2f) * (DOCSP-32596): Set up CORS on the server (#103) (76c9530) * (DOCSP-32104): Do not serve demo site in prod environment (#101) (5f0ea11) * (DOCSP-32247): Fix preprocessed content not added to DB (#100) (57cc24e) * fix handling of openapi specs (#99) (6f9b0e4) * fix hanging promise in async recursive operation (#98) (c974058) * Drone fix (#97) (e54a272) * (DOCSP-32363) [UI] handle cases where the LLM stops mid code example (#85) (9e26c7f) * (DOCSP-32155) [UI] UX Feedback (#81) (c34b246) * (DOCSP-32243) Handle and test pages with page-level code block select (#94) (f6c3f7e) * (DOCSP-32490) [UI] Link to current commit on GitHub (#90) (64dbd0d) * (DOCSP-32242): Handle tabbed Snooty content (#92) (f87f67d) * Restore 'tags' and use 'tagsIn' only internally (#91) (035f018) * (DOCSP-32227): Improve OpenAPI spec ingestion (#88) (f369218) * (DOCSP-31343): Add system diagram and info for ingest (#86) (6e4402b) * (DOCSP-32362): Improve conversation request logging (#89) (b251e0d) * (DOCSP-32253): Add more semantically relevant product names to metadata (#83) (5869f22) * (DOCSP-32434) [UI] Add error text for non-modal input (#87) (5de9282) * (DOCSP-32206): Strip comments from rst -> md (#84) (611a50d) * (DOCSP-32194): Fix drone and k8s for ingest/chat-server staging (#80) (b61c9bc) * Define engines to remove EBADENGINE warning (#82) (f333ef5) * (DOCSP-31440, DOCSP-32075): Add tags to chunk metadata (#78) (254ce2b) * (DOCSP-32217): Change chunker to chunk based on number of tokens + add max context tokens per message (#79) (e7c1cb6) * (DOCSP-32183 & DOCSP-32226): Automate test creation based on YAML file (#77) (30eb660) * Add database index creation scripts (#75) (53f296f) * (DOCSP-31575): LLM qualitative testing framework (#74) (f222e54) * (DOCSP-31111) [UI] publish react component to npm (#73) (3baf40e) * Refactor `ConversationService` class as `makeConversationService` func (#76) (6cb9a83) * (DOCSP-32070): LLM preprocessing on user queries (#72) (6ff4219) * (DOCSP-31925): Support tables in parsed snooty MD (#67) (539d403) * Reduce ChatGPT API flakiness + faster timeout when flaky (#69) (8938599) * Update Chat UI Packaging (#71) (2e5a0fb) * (DOCSP-32066): Q&A server reduce amount of backoff on embedding (#68) (fe938b0) * (DOCSP-31639): fix timeout err in tests (#70) (741ea29) * Implement page titles and prepending frontmatter to chunks (#64) (cf56a26) * Fix JS error surfaced in the UI (#66) (5e6866a) * bundle LG Chat with component lib (c4fc213) * Modify ui config for pre-release (135d211) * (DOCSP-31154) [UI] Handle timeout & "conversation too long" errors (#65) (1943608) * (DOCSP-31791): Boost manual in the results for short queries + make app config file (#63) (6449993) * (DOCSP-31523) [UI] Replace custom components with LG Chat (#61) (3ee640b) * Fix ingest drone + err handling (#59) (8307750) * Fix code block lang (#62) (cffcbf0) * Get current branch from Snooty Data API plus allow override (#56) (6114e7c) * Clean up pages for embedding content (#60) (45ce277) * (DOCSP-31799): Parse definition lists (and other relevant content inside of `"target"` nodes) (#58) (2d7d1b3) * rename chat-ui package (#57) (dc7794f) * (DOCSP-31574): Create qualitative testing env (#55) (0a1c92a) * Add tests for data deletion (#48) (9b1417a) * Update Snooty Data Source to strip 'index' (#50) (d6b0b2f) * Add ability to override Snooty project baseUrl (#52) (fccc8ca) * Split up tests + add utils (#53) (0dedc6f) * (DOCSP-31436) [Q&A] Investigate & improve streaming rate (#54) (cc90cd0) * (DOCSP-31120) [Q&A] separate further reading reference links from LLM response (#43) (a604d83) * (DOCSP-30863) [Q&A] type check requests (#23) (5f908c0) * Streamline/nitpick/bikeshed the PR template (#51) (28b4e1b) * (DOCSP-31601): Update server base URL (#49) (1c60a0b) * (DOCSP-31573): Update prompts to address bugs (#47) (a366803) * (DOCSP-30602): Ingest service prod pipeline (#42) (8e60755) * Update pull_request_template.md (ab166a7) * Monorepo docker build fix (#46) (1dcef28) * (DOCSP-31284): Integrate prod Snooty Data API (#44) (f80ce51) * (DOCSP-31360) [UI] Conditional Streaming (#45) (34490bb) * update root package-lock (578b06b) * rename chat-ui package (d7b6dcf) * Add shared dev script to monorepo root (67811d1) * Streaming works (#25) (c173ba2) * (DOCSP-30601): Create production release system (#41) (7472c9d) * (DOCSP-31304): Implement 'all' command (#40) (f0c4cf2) * (DOCSP-31201) [UI] Updated Legalese (#39) (39c0f97) * (DOCSP-30584): ingest service CI testing and staging drone pipeline (#38) (d2e8325) * Fix + improve devcenter source tests (#37) (be0a5eb) * Fix URL building for devcenter paths (#36) (4c75175) * Add logic for ingest resumption (#35) (0f650c2) * stricter LLM to reduce halucination risk + change demo page title (#33) (cfb4628) * (DOCSP-30890) [UI] Make component mobile responsive (#29) (66080a7) * Add minor optimization for rate-limited embed requests (#34) (7547e31) * (DOCSP-31121): Add request tracking across logs (#32) (7fdc1e5) * (DOCSP-30988): Implement ingest E2E (#28) (8467989) * (DOCSP-30843): IP Address tracking and validation (#21) (7335672) * Update seed-data script (4f7a2f7) * (DOCSP-31050): Fix CI tests + run tests on PR (#30) (74fe090) * (DOCSP-30953 & DOCSP-30945): Error and edge case handling (#24) (af1b845) * Add PR template to repo (#26) (5231ef0) * Fix tests (#27) (58599ea) * more specific linter targets (18d2f5d) * Move and refactor EmbeddedContent/Store and clean up core services (#19) (43078a2) * separate format from lint (3a817b7) * remove console logs (#22) (20027ce) * Add lists to LGMarkdown (#20) (14cf986) * (DOCSP-30964): Staging build (#18) (1163ff6) * Fix renames (945337e) * Rename "chunks" to "embedded content" (#17) (a18bb8d) * Implement Data Sources (#12) (e9c5784) * Move reusable pieces to chat-core (#14) (3babf35) * (DOCSP-30577): POST `/conversations/:conversationId/messages/:messageId/rating` endpoint (#16) (c659173) * E2E (#15) (c85a2aa) * failed message handling (16acff7) * better prompt validation error handling (6408669) * UI tweaks (f803938) * (DOCSP-30921) [UI] Add prompt validation & error handling (1) (cb98ec7) * (DOCSP-30616): `POST /conversations/:conversationId/messages/` route, no streaming (#13) (8451c08) * Add publishing pipeline (42ab17e) * (DOCSP-30575): POST /conversations/ route (#11) (d609171) * (DOCSP-30619, DOCSP-30758, DOCSP-30722): Add MongoDB integration and conversation/content services (#9) (020a501) * Implement ingest store/database (#8) (5b9f8f6) * Add initial design docs (#1) (31d9069) * (DOCSP-30585): Generate search index script (#10) (458403f) * Merge branch 'demodalize' of github.com:nlarew/docs-chatbot (8d46f37) * better avatars (4c7c712) * tweak card heading (40d1dff) * loading ui (58256da) * (DOCSP-30617): OpenAI LLM service (#7) (d4af1f9) * refactor + loading states (4418f8d) * (DOCSP-30618): Create embeddings service (#6) (c54d123) * mock out Q&A API (9d80c20) * Refactor to use a single chat input (6ac351a) * Animation works except for Card ref (2bc2b41) * Rename & refactor (b5385af) * Send button for all inputs with text (9ca1873) * Move input state up (664b456) * Animation works (b8c2f42) * Use card modal WIP (27c2392) * cleanup (d96bcf0) * Fix bad imports (524f896) * (DOCSP-30576): Stub out Q&A server (#4) (899c0a3) * LGMarkdown (33596cf) * Working on LGMarkdown (897b141) * consistent models with backend (319b2ae) * UI lint pass (d8051d9) * Leafie avatar (ebd660b) * Don't require React Context for conversations (86c90b6) * fix modal style (6ae1500) * Add storybook (73a4ec3) * Modal (cac03d2) * fix banner width (088d858) * Merge pull request #3 from mongodb/DOCSP-30565 (5264da5) * Update .drone.yml (a9e3ba5) * working staging build + updated untested prod (fe16f6c) * remove uncreated env secrets (318c501) * fix broken registry path (a1e1dff) * change release (072100f) * update environment info (ff22f4f) * empty state (a66ce15) * fix value file location (ee4f06b) * fix pipeline (89f9e8b) * Trigger build (408d7dc) * Add (broken) tests (329d9e0) * Trigger build (6cae68f) * Fix context (7d213c0) * update path to docker file (f0edac3) * trigger build (e5e64c0) * more debugging (2dfb919) * Use dependency injection for PageStore/ChunkStore interfaces (89577b8) * Remove start comment (287182d) * move drone file (7453f99) * trigger build? (95579f6) * test deploy pipeline (aab4632) * add cta (db01cdb) * apply prettier (c49d368) * remove unused imports (85f6ab3) * add prettier (c1cf166) * combined props (ae9d66c) * chat state works (e29ee25) * add temp aria-labelledby prop (a0d4695) * Scaffold the ingest CLI (81a5c41) * Update ingest/README (65d2902) * chatbot-ui -> chat-ui (4340d1a) * Merge pull request #2 from mongodben/DOCSP-30486 (17662e6) * Bootstrap ingest CLI tool (0590da2) * Add chatbot-ui (997bec5) * set up QA server project structure (7cd25d5) * Delete specs directory (6b06d68) * ready repo for first push (2b250da)

mongodben added 8 commits August 23, 2023 10:48

update drone file for testing

4f83796

add trigger

836b1d0

fix and trigger

8c6dea8

fix and trigger

daef47e

trigger chat server build

cef4a6d

cean up tmp changes

91a13d8

remove triggers

6db791f

fix handing promise

7692e7c

cbush approved these changes Aug 23, 2023

View reviewed changes

mongodben merged commit e54a272 into main Aug 23, 2023

mongodben deleted the drone_fix branch August 23, 2023 18:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Drone fix #97

Drone fix #97

mongodben commented Aug 23, 2023 •

edited

Loading

Drone fix #97

Drone fix #97

Conversation

mongodben commented Aug 23, 2023 • edited Loading

Changes

Notes

mongodben commented Aug 23, 2023 •

edited

Loading