[NIGHTLY] v24.10.00

🔗 Links

• Development Branch
• Compare with main branch

🚨 Breaking Changes

• Finalize removing MultiMessage from Morpheus (#1886) @yczhang-nv
• Removing support for MultiMessage from stages (#1803) @yczhang-nv

🐛 Bug Fixes

• Fixing the docker build when Morpheus is a submodule (#1914) @mdemoret-nv
• Fix conda path for missing llm packages (#1907) @dagardner-nv
• Update WriteToVectorDBStage to re-raise errors from the underlying database (#1905) @dagardner-nv
• Avoid memory leak warnings from pypdfium2 (#1902) @dagardner-nv
• Remove redundant copy of the load_labels_file method (#1901) @dagardner-nv
• Fix Can't find 'action.yml' CI error (#1896) @dagardner-nv
• Fix DFP integrated training Azure pipeline (#1894) @yczhang-nv
• Drop 'CI Pipeline / Check' dependency from the 'package-core' job (#1885) @AnuradhaKaruppiah
• Python source stages now optionally receive a reference to mrc.Subscription (#1881) @dagardner-nv
• Fix Unregistered type : mrc::pymrc::coro::BoostFibersMainPyAwaitable error (#1869) @dagardner-nv
• Revert PR_1736 "Always run the PR builder step even if others are cancelled" (#1860) @AnuradhaKaruppiah
• ensure columns are strings before concatenation (#1857) @cwharris
• Update Kafka DL script to 2.13-3.8.0 (#1856) @cwharris
• Update isort settings file path in fix_all.sh (#1855) @yczhang-nv
• Move isort settings into pyproject.toml (#1854) @dagardner-nv
• Update location of morpheus setup and data files in VS settings (#1843) @AnuradhaKaruppiah
• Fix isort config marking _utils as known first party (#1842) @dagardner-nv
• Fix usage of the C++ impl of write_df_to_file (#1840) @dagardner-nv
• Fix shutdown on Ctrl+C for Python source stages (#1839) @dagardner-nv
• Improved type-hints for stage and source decorators (#1831) @dagardner-nv
• Add tests to confirm that a mis-configured MultiPortModulesStage will raise an exception rather than segfaulting (#1829) @dagardner-nv
• Ensure proper initialization of CMAKE_INSTALL_PREFIX if needed (#1815) @dagardner-nv
• Fix merging of CLI args and Yaml configs in vdb_upload example (#1813) @dagardner-nv
• Fix log parsing undefined variable and duplicate sequence id errors (#1809) @dagardner-nv
• Remove obsolete version string from compose yamls (#1808) @dagardner-nv
• Ensure the release container does not contain any unintended files (#1807) @dagardner-nv
• Update ci/release/update-version.sh to include missed files (#1801) @dagardner-nv

📖 Documentation

• Remove TODO statements from documentation (#1879) @dagardner-nv
• Remove automock for merlin as we no longer have/use merlin (#1830) @dagardner-nv
• Add documentation checks to CI (#1821) @dagardner-nv
• Fix documentation links to work in both source repo and documentation builds (#1814) @dagardner-nv
• Update documentation for vdb_upload to use realistic source data with the --file_source flag (#1800) @dagardner-nv

🚀 New Features

• Move vector db stages to morpheus-llm (#1889) @AnuradhaKaruppiah
• Scripts for building and uploading the morpheus-core conda package (#1883) @AnuradhaKaruppiah
• Implement MultiProcessingStage (#1878) @yczhang-nv
• [morpheus-refactor] Move morpheus source to python/morpheus (#1836) @AnuradhaKaruppiah
• Remove NVTabular (#1825) @cwharris
• Create a Docker image for Morpheus models (#1804) @dagardner-nv
• Add unique column to output of the log_parsing pipeline (#1795) @dagardner-nv

🛠️ Improvements

• Finalize removing MultiMessage from Morpheus (#1886) @yczhang-nv
• Run pre-commit on all files, not just python (#1880) @cwharris
• Prefer len(os.sched_getaffinity(0)) over os.cpu_count() (#1866) @cwharris
• Remove cloudtrail debug log from autoencoder source stage (#1865) @cwharris
• Run yapf, flake8, isort as part of pre-commit (#1859) @cwharris
• Warn when Config's pipeline_batch_size < model_max_batch_size (#1858) @cwharris
• Breakout morpheus_llm (#1853) @AnuradhaKaruppiah
• Install built documentation into release container (#1806) @dagardner-nv
• Removing support for MultiMessage from stages (#1803) @yczhang-nv
• Batch incoming DOCA raw packet data (#1731) @dagardner-nv