Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrap raw types into a StageSchema class #7

Conversation

dagardner-nv
Copy link
Owner

No description provided.

dagardner-nv and others added 30 commits September 29, 2023 14:07
…s#630)

Adds support for querying data from Databricks delta tables and write the results to delta tables. Resolves nv-morpheus#611

Authors:
  - https://github.com/pthalasta

Approvers:
  - Christopher Harris (https://github.com/cwharris)
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: nv-morpheus#630
…pheus#1243)

- Updated SQL loader to utilize connections from the pool.
- Fixed control message utility variable referenced before assignment error.
closes nv-morpheus#1237 nv-morpheus#1235

Authors:
  - Bhargav Suryadevara (https://github.com/bsuryadevara)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: nv-morpheus#1243
…rge-23.07

Forward-merge branch-23.07 to branch-23.11
dagardner-nv and others added 28 commits October 6, 2023 13:29
…avid-pipe-build-refactor-stage-schema [no ci]
* Silence deprecation warnings from SimpleAmqpClient nv-morpheus#1255 

fixes nv-morpheus#1185

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Devin Robison (https://github.com/drobison00)

URL: nv-morpheus#1186
* Set nodejs version to 18.*
Closes nv-morpheus#1259

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: nv-morpheus#1262
…ge-23.07

Resolve merge conflicts between 23.07 and 23.11
Add `@wraps` decorator to `cm_skip_processing_if_failed` to allow docstrings for wrapped methods to generate documentation.

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - David Gardner (https://github.com/dagardner-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: nv-morpheus#1264
…Bs (nv-morpheus#1225)

Closes nv-morpheus#1177 

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Bhargav Suryadevara (https://github.com/bsuryadevara)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: nv-morpheus#1225
…#1256)

- Replace `requirements.txt`files (pip) with `requirements.yml` (conda environment).
- Update dependency versions.

Closes nv-morpheus#1226
Closes nv-morpheus#1227

## By Submitting this PR I confirm:
- I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md).
- When the PR is ready for review, new or existing tests cover these changes.
- When the PR is ready for review, the documentation is up to date with these changes.

Authors:
  - Eli Fajardo (https://github.com/efajardo-nv)

Approvers:
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: nv-morpheus#1256
dagardner-nv pushed a commit that referenced this pull request Dec 19, 2023
I have a few changes here. Some of them might not make it depending on how others feel.

- Move test utilities in to a library to share across tests.
- Move tests in to individual executable to enforce individual tests to not sharing code.
- Use `gtest_discover_tests` to let `ctest` run and show all tests individually.
- Use `gtest::gtest_main` and remove explicit main entry file.
- ~Build and run tests by default.~
- Ability to filter tests by `ctest --test-dir build -R "TestCuda.*"`

```
+ echo 'Running CTest...'
Running CTest...
+ ctest --test-dir build
Internal ctest changing into directory: /workspaces/morpheus/build
Test project /workspaces/morpheus/build
      Start  1: TestCuda.LargeShape
 1/35 Test  #1: TestCuda.LargeShape .......................................   Passed    0.17 sec
      Start  2: TestDataLoader.DataLoaderInitializationTest
 2/35 Test  #2: TestDataLoader.DataLoaderInitializationTest ...............   Passed    0.07 sec
      Start  3: TestDataLoader.DataLoaderRegisterLoaderTest
 3/35 Test  #3: TestDataLoader.DataLoaderRegisterLoaderTest ...............   Passed    0.07 sec
      Start  4: TestDataLoader.DataLoaderRemoveLoaderTest
 4/35 Test  #4: TestDataLoader.DataLoaderRemoveLoaderTest .................   Passed    0.07 sec
      Start  5: TestDataLoader.PayloadLoaderTest
 5/35 Test  #5: TestDataLoader.PayloadLoaderTest ..........................   Passed    1.42 sec
      Start  6: TestDataLoader.FileLoaderTest
 6/35 Test  #6: TestDataLoader.FileLoaderTest .............................   Passed    2.65 sec
      Start  7: TestControlMessage.InitializationTest
 7/35 Test  #7: TestControlMessage.InitializationTest .....................   Passed    0.07 sec
      Start  8: TestControlMessage.SetMessageTest
 8/35 Test  #8: TestControlMessage.SetMessageTest .........................   Passed    0.07 sec
      Start  9: TestControlMessage.TaskTest
 9/35 Test  #9: TestControlMessage.TaskTest ...............................   Passed    0.07 sec
      Start 10: TestControlMessage.PayloadTest
10/35 Test #10: TestControlMessage.PayloadTest ............................   Passed    1.45 sec
      Start 11: TestDataLoaderModule.DataLoaderModuleInitializationTest
11/35 Test nv-morpheus#11: TestDataLoaderModule.DataLoaderModuleInitializationTest ...   Passed    0.07 sec
      Start 12: TestDataLoaderModule.EndToEndPayloadDataLoaderTest
12/35 Test nv-morpheus#12: TestDataLoaderModule.EndToEndPayloadDataLoaderTest ........   Passed    0.12 sec
      Start 13: TestDeserializers.GetIndexColCountNoIdxFromFile
13/35 Test nv-morpheus#13: TestDeserializers.GetIndexColCountNoIdxFromFile ...........   Passed    1.39 sec
      Start 14: TestDeserializers.GetIndexColCountWithIdxFromFile
14/35 Test nv-morpheus#14: TestDeserializers.GetIndexColCountWithIdxFromFile .........   Passed    1.37 sec
      Start 15: TestDeserializers.GetIndexColCountNoIdxSimilarName
15/35 Test nv-morpheus#15: TestDeserializers.GetIndexColCountNoIdxSimilarName ........   Passed    0.19 sec
      Start 16: TestDeserializers.GetIndexColCountIdx
16/35 Test nv-morpheus#16: TestDeserializers.GetIndexColCountIdx .....................   Passed    0.18 sec
      Start 17: TestDeserializers.GetIndexColCountValidNameInvalidType
17/35 Test nv-morpheus#17: TestDeserializers.GetIndexColCountValidNameInvalidType ....   Passed    0.16 sec
      Start 18: TestDevMemInfo.RmmBufferConstructor
18/35 Test nv-morpheus#18: TestDevMemInfo.RmmBufferConstructor .......................   Passed    0.16 sec
      Start 19: TestDevMemInfo.VoidPtrConstructor
19/35 Test nv-morpheus#19: TestDevMemInfo.VoidPtrConstructor .........................   Passed    0.25 sec
      Start 20: TestDevMemInfo.MakeNewBuffer
20/35 Test nv-morpheus#20: TestDevMemInfo.MakeNewBuffer ..............................   Passed    0.25 sec
      Start 21: TestFileInOut.RoundTripCSV
21/35 Test nv-morpheus#21: TestFileInOut.RoundTripCSV ................................   Passed    2.67 sec
      Start 22: TestFileInOut.RoundTripJSON
22/35 Test nv-morpheus#22: TestFileInOut.RoundTripJSON ...............................   Passed    2.69 sec
      Start 23: TestMatxUtil.ReduceMax1d
23/35 Test nv-morpheus#23: TestMatxUtil.ReduceMax1d ..................................   Passed    0.17 sec
      Start 24: TestMatxUtil.ReduceMax2dRowMajor
24/35 Test nv-morpheus#24: TestMatxUtil.ReduceMax2dRowMajor ..........................   Passed    0.17 sec
      Start 25: TestMatxUtil.ReduceMax2dColMajor
25/35 Test nv-morpheus#25: TestMatxUtil.ReduceMax2dColMajor ..........................   Passed    1.40 sec
      Start 26: TestMatxUtil.Cast
26/35 Test nv-morpheus#26: TestMatxUtil.Cast .........................................   Passed    0.16 sec
      Start 27: TestMatxUtil.Threshold
27/35 Test nv-morpheus#27: TestMatxUtil.Threshold ....................................   Passed    0.16 sec
      Start 28: TestMatxUtil.ThresholdByRow
28/35 Test nv-morpheus#28: TestMatxUtil.ThresholdByRow ...............................   Passed    0.19 sec
      Start 29: TestMultiSlices.Ranges
29/35 Test nv-morpheus#29: TestMultiSlices.Ranges ....................................   Passed    1.37 sec
      Start 30: TestTensor.UtilsShapeString
30/35 Test nv-morpheus#30: TestTensor.UtilsShapeString ...............................   Passed    0.06 sec
      Start 31: TestTensor.GetElementStride
31/35 Test nv-morpheus#31: TestTensor.GetElementStride ...............................   Passed    0.06 sec
      Start 32: TestTensor.AsType
32/35 Test nv-morpheus#32: TestTensor.AsType .........................................   Passed    0.16 sec
      Start 33: TestTensor.Create
33/35 Test nv-morpheus#33: TestTensor.Create .........................................   Passed    0.16 sec
      Start 34: TestTensor.UtilsValidateShapeAndStride
34/35 Test nv-morpheus#34: TestTensor.UtilsValidateShapeAndStride ....................   Passed    0.16 sec
      Start 35: TestTypeUtils.DTypeCopy
35/35 Test nv-morpheus#35: TestTypeUtils.DTypeCopy ...................................   Passed    0.06 sec

100% tests passed, 0 tests failed out of 35

Total Test time (real) =  19.91 sec
```

Authors:
  - Christopher Harris (https://github.com/cwharris)

Approvers:
  - David Gardner (https://github.com/dagardner-nv)

URL: nv-morpheus#1095
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants