Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement C++ version of MonitorStage #1908

Open
wants to merge 51 commits into
base: branch-25.02
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
201aa0b
draft pr for dependency
yczhang-nv Sep 24, 2024
a43c063
Initial commit for monitor stage
yczhang-nv Sep 24, 2024
710b68a
add monitor controller
yczhang-nv Sep 24, 2024
e2a594e
fix headers
yczhang-nv Sep 24, 2024
1ab2b3e
Merge branch 'branch-24.10' into impl-cpp-monitor-stage
yczhang-nv Sep 25, 2024
0ef38d5
improve monitor_controller
yczhang-nv Sep 25, 2024
e1a95d3
Merge branch 'impl-cpp-monitor-stage' of github.com:yczhang-nv/Morphe…
yczhang-nv Sep 25, 2024
52ccfbe
reorg files
yczhang-nv Sep 25, 2024
83c5b35
initial commit for unit tests
yczhang-nv Sep 25, 2024
b0027c1
template code should be in the header files
yczhang-nv Sep 25, 2024
547f276
update test
yczhang-nv Sep 26, 2024
9847952
fix monitor controller impl & unit test
yczhang-nv Sep 27, 2024
c23943e
Try to fix segfault
yczhang-nv Sep 30, 2024
72d866c
single monitor now works
yczhang-nv Oct 1, 2024
4528dae
Either multi bars no logs, or single bar with logs
yczhang-nv Oct 1, 2024
47549f3
Make C++ stage work in Python
yczhang-nv Oct 1, 2024
896d68c
Merge remote-tracking branch 'upstream/branch-24.10' into impl-cpp-mo…
yczhang-nv Oct 1, 2024
75f3d23
Creates unit test
yczhang-nv Oct 1, 2024
4223f4e
fixed everything!
yczhang-nv Oct 1, 2024
3d61300
Fix CI
yczhang-nv Oct 1, 2024
ad4d312
Merge remote-tracking branch 'upstream/branch-24.10' into impl-cpp-mo…
yczhang-nv Oct 1, 2024
18202c7
fix dependencies
yczhang-nv Oct 1, 2024
85e048f
fix CI
yczhang-nv Oct 2, 2024
e506b95
Fix documentation
yczhang-nv Oct 2, 2024
51d1fff
Fix CI
yczhang-nv Oct 2, 2024
0b6aae0
Merge branch 'branch-24.10' into impl-cpp-monitor-stage
yczhang-nv Oct 3, 2024
c606e70
pretty close but the impl is a mess
yczhang-nv Oct 5, 2024
1fce0be
Merge branch 'impl-cpp-monitor-stage' of github.com:yczhang-nv/Morphe…
yczhang-nv Oct 5, 2024
48cea1f
Multiple line keep moving downwards but log message does not display
yczhang-nv Oct 7, 2024
be0c749
Displays log but log itself covers older logs
yczhang-nv Oct 7, 2024
7dc2936
This works
yczhang-nv Oct 8, 2024
3d067a9
add Python bindings
yczhang-nv Oct 8, 2024
1aba9d5
fix format
yczhang-nv Oct 8, 2024
4a1797e
Merge remote-tracking branch 'upstream/branch-24.10' into impl-cpp-mo…
yczhang-nv Oct 8, 2024
2f75d1a
update dependency
yczhang-nv Oct 8, 2024
a8f87f0
fix CI
yczhang-nv Oct 8, 2024
a413c47
fix CI
yczhang-nv Oct 8, 2024
d5ce47b
fix doc
yczhang-nv Oct 8, 2024
51eed80
fix format
yczhang-nv Oct 8, 2024
86ecdce
update test_monitor_stage_pipe to accelerate testing
yczhang-nv Oct 8, 2024
cfe99db
Merge remote-tracking branch 'upstream/branch-24.10' into impl-cpp-mo…
yczhang-nv Oct 15, 2024
adeab7b
Update comments
yczhang-nv Oct 15, 2024
4b39de3
fix format
yczhang-nv Oct 15, 2024
3a54e6b
fix types
yczhang-nv Oct 15, 2024
afc3c0c
fix format
yczhang-nv Oct 15, 2024
b565708
Merge branch 'branch-24.10' into impl-cpp-monitor-stage
yczhang-nv Oct 16, 2024
585dcd0
Merge remote-tracking branch 'upstream/branch-25.02' into impl-cpp-mo…
yczhang-nv Nov 6, 2024
2d168b2
Merge remote-tracking branch 'upstream/branch-25.02' into impl-cpp-mo…
yczhang-nv Nov 6, 2024
279df4a
update utilities
yczhang-nv Nov 6, 2024
06e72ec
fix tests
yczhang-nv Nov 6, 2024
4521b6a
fix format
yczhang-nv Nov 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions cmake/dependencies_core.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ list(APPEND CMAKE_MESSAGE_CONTEXT "dep_core")
# =========
morpheus_utils_configure_cccl()

# indicators
# ==========
morpheus_utils_configure_indicators()

# matx
# ====
morpheus_utils_configure_matx()
Expand Down
1 change: 1 addition & 0 deletions conda/environments/all_cuda-125_arch-x86_64.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ dependencies:
- gxx=12.1
- huggingface_hub=0.20.2
- include-what-you-use=0.20
- indicators=2.3
- ipython
- isort
- jsonpatch>=1.33
Expand Down
1 change: 1 addition & 0 deletions conda/environments/dev_cuda-125_arch-x86_64.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ dependencies:
- gtest=1.14
- gxx=12.1
- include-what-you-use=0.20
- indicators=2.3
- ipython
- isort
- libcublas-dev
Expand Down
1 change: 1 addition & 0 deletions dependencies.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,7 @@ dependencies:
- cmake=3.27
- cuda-cudart-dev=12.5
- cuda-version=12.5
- indicators=2.3 # C++ library for displaying progress bars
- libtool # Needed for DOCA build
- ninja=1.11
- pkg-config=0.29 # for mrc cmake
Expand Down
1 change: 1 addition & 0 deletions python/morpheus/morpheus/_lib/cmake/libmorpheus.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ add_library(${PROJECT_NAME}::morpheus ALIAS morpheus)

target_link_libraries(morpheus
PRIVATE
indicators::indicators
matx::matx
$<$<CONFIG:Debug>:ZLIB::ZLIB>
PUBLIC
Expand Down
103 changes: 103 additions & 0 deletions python/morpheus/morpheus/_lib/common/__init__.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ __all__ = [
"FilterSource",
"HttpEndpoint",
"HttpServer",
"IndicatorsFontStyle",
"IndicatorsTextColor",
"Tensor",
"TypeId",
"determine_file_type",
Expand Down Expand Up @@ -120,6 +122,107 @@ class HttpServer():
def start(self) -> None: ...
def stop(self) -> None: ...
pass
class IndicatorsFontStyle():
"""
Members:

bold

dark

italic

underline

blink

reverse

concealed

crossed
"""
def __eq__(self, other: object) -> bool: ...
def __getstate__(self) -> int: ...
def __hash__(self) -> int: ...
def __index__(self) -> int: ...
def __init__(self, value: int) -> None: ...
def __int__(self) -> int: ...
def __ne__(self, other: object) -> bool: ...
def __repr__(self) -> str: ...
def __setstate__(self, state: int) -> None: ...
@property
def name(self) -> str:
"""
:type: str
"""
@property
def value(self) -> int:
"""
:type: int
"""
__members__: dict # value = {'bold': <IndicatorsFontStyle.bold: 0>, 'dark': <IndicatorsFontStyle.dark: 1>, 'italic': <IndicatorsFontStyle.italic: 2>, 'underline': <IndicatorsFontStyle.underline: 3>, 'blink': <IndicatorsFontStyle.blink: 4>, 'reverse': <IndicatorsFontStyle.reverse: 5>, 'concealed': <IndicatorsFontStyle.concealed: 6>, 'crossed': <IndicatorsFontStyle.crossed: 7>}
blink: morpheus._lib.common.IndicatorsFontStyle # value = <IndicatorsFontStyle.blink: 4>
bold: morpheus._lib.common.IndicatorsFontStyle # value = <IndicatorsFontStyle.bold: 0>
concealed: morpheus._lib.common.IndicatorsFontStyle # value = <IndicatorsFontStyle.concealed: 6>
crossed: morpheus._lib.common.IndicatorsFontStyle # value = <IndicatorsFontStyle.crossed: 7>
dark: morpheus._lib.common.IndicatorsFontStyle # value = <IndicatorsFontStyle.dark: 1>
italic: morpheus._lib.common.IndicatorsFontStyle # value = <IndicatorsFontStyle.italic: 2>
reverse: morpheus._lib.common.IndicatorsFontStyle # value = <IndicatorsFontStyle.reverse: 5>
underline: morpheus._lib.common.IndicatorsFontStyle # value = <IndicatorsFontStyle.underline: 3>
pass
class IndicatorsTextColor():
"""
Members:

grey

red

green

yellow

blue

magenta

cyan

white

unspecified
"""
def __eq__(self, other: object) -> bool: ...
def __getstate__(self) -> int: ...
def __hash__(self) -> int: ...
def __index__(self) -> int: ...
def __init__(self, value: int) -> None: ...
def __int__(self) -> int: ...
def __ne__(self, other: object) -> bool: ...
def __repr__(self) -> str: ...
def __setstate__(self, state: int) -> None: ...
@property
def name(self) -> str:
"""
:type: str
"""
@property
def value(self) -> int:
"""
:type: int
"""
__members__: dict # value = {'grey': <IndicatorsTextColor.grey: 0>, 'red': <IndicatorsTextColor.red: 1>, 'green': <IndicatorsTextColor.green: 2>, 'yellow': <IndicatorsTextColor.yellow: 3>, 'blue': <IndicatorsTextColor.blue: 4>, 'magenta': <IndicatorsTextColor.magenta: 5>, 'cyan': <IndicatorsTextColor.cyan: 6>, 'white': <IndicatorsTextColor.white: 7>, 'unspecified': <IndicatorsTextColor.unspecified: 8>}
blue: morpheus._lib.common.IndicatorsTextColor # value = <IndicatorsTextColor.blue: 4>
cyan: morpheus._lib.common.IndicatorsTextColor # value = <IndicatorsTextColor.cyan: 6>
green: morpheus._lib.common.IndicatorsTextColor # value = <IndicatorsTextColor.green: 2>
grey: morpheus._lib.common.IndicatorsTextColor # value = <IndicatorsTextColor.grey: 0>
magenta: morpheus._lib.common.IndicatorsTextColor # value = <IndicatorsTextColor.magenta: 5>
red: morpheus._lib.common.IndicatorsTextColor # value = <IndicatorsTextColor.red: 1>
unspecified: morpheus._lib.common.IndicatorsTextColor # value = <IndicatorsTextColor.unspecified: 8>
white: morpheus._lib.common.IndicatorsTextColor # value = <IndicatorsTextColor.white: 7>
yellow: morpheus._lib.common.IndicatorsTextColor # value = <IndicatorsTextColor.yellow: 3>
pass
class Tensor():
@staticmethod
def from_cupy(arg0: object) -> Tensor: ...
Expand Down
23 changes: 23 additions & 0 deletions python/morpheus/morpheus/_lib/common/module.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@
#include <pybind11/pybind11.h>
#include <pybind11/pytypes.h> // for return_value_policy::reference
// for pathlib.Path -> std::filesystem::path conversions
#include <indicators/color.hpp>
#include <indicators/font_style.hpp>
#include <pybind11/stl.h> // IWYU pragma: keep
#include <pybind11/stl/filesystem.h> // IWYU pragma: keep

Expand Down Expand Up @@ -169,6 +171,27 @@ PYBIND11_MODULE(common, _module)
.def("__enter__", &HttpServerInterfaceProxy::enter, py::return_value_policy::reference)
.def("__exit__", &HttpServerInterfaceProxy::exit);

py::enum_<indicators::Color>(_module, "IndicatorsTextColor")
.value("grey", indicators::Color::grey)
.value("red", indicators::Color::red)
.value("green", indicators::Color::green)
.value("yellow", indicators::Color::yellow)
.value("blue", indicators::Color::blue)
.value("magenta", indicators::Color::magenta)
.value("cyan", indicators::Color::cyan)
.value("white", indicators::Color::white)
.value("unspecified", indicators::Color::unspecified);

py::enum_<indicators::FontStyle>(_module, "IndicatorsFontStyle")
.value("bold", indicators::FontStyle::bold)
.value("dark", indicators::FontStyle::dark)
.value("italic", indicators::FontStyle::italic)
.value("underline", indicators::FontStyle::underline)
.value("blink", indicators::FontStyle::blink)
.value("reverse", indicators::FontStyle::reverse)
.value("concealed", indicators::FontStyle::concealed)
.value("crossed", indicators::FontStyle::crossed);

_module.attr("__version__") =
MRC_CONCAT_STR(morpheus_VERSION_MAJOR << "." << morpheus_VERSION_MINOR << "." << morpheus_VERSION_PATCH);
}
Expand Down
Loading
Loading