Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Root cause analysis datasets #470

Merged
8 commits merged into from
Nov 18, 2022
Merged

Root cause analysis datasets #470

8 commits merged into from
Nov 18, 2022

Conversation

efajardo-nv
Copy link
Contributor

@efajardo-nv efajardo-nv commented Nov 16, 2022 •
edited
Loading

  • Add training data used by root cause analysis training script and notebook
  • Replace existing CSV validation data file with JSON lines
  • Update root cause pipeline command in README to read and output JSON lines

Unblocks #452

@efajardo-nv efajardo-nv added non-breaking Non-breaking change improvement Improvement to existing functionality labels Nov 16, 2022
@efajardo-nv efajardo-nv requested review from a team as code owners November 16, 2022 18:51
raykallen
raykallen approved these changes Nov 16, 2022
View reviewed changes
Copy link
Contributor

@raykallen raykallen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked all data files. They correctly match the small anonymized kernel logs to be shared publicly.

@efajardo-nv efajardo-nv mentioned this pull request Nov 17, 2022
Merged
mdemoret-nv
mdemoret-nv reviewed Nov 17, 2022
View reviewed changes
Copy link
Contributor

@mdemoret-nv mdemoret-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace existing CSV validation data file with JSON lines

Whats the reason for converting from CSV to JSON lines? CSV generally works much better with Morpheus as it's easier to parse and C++ accelerated.

@efajardo-nv
Copy link
Contributor Author

efajardo-nv commented Nov 17, 2022 •
edited
Loading

@mdemoret-nv we decided to switch to jsonlines for consistency since it's also used in phishing and sid-nlp as well as @gbatmaz's root cause inference script.

ghost pushed a commit that referenced this pull request Nov 18, 2022
@gbatmaz
Root cause use case scripts (#452)
c22d21d 
Raising this PR for root cause use case. Data is pending approval.

Depends on  #470 (Root cause analysis datasets]
Closes #453

Authors:
  - https://github.com/gbatmaz
  - Eli Fajardo (https://github.com/efajardo-nv)

Approvers:
  - https://github.com/raykallen

URL: #452
@raykallen
Copy link
Contributor

@gpucibot merge

@ghost ghost merged commit 1e4b717 into nv-morpheus:branch-22.11 Nov 18, 2022
@efajardo-nv efajardo-nv deleted the rootcause-data branch July 29, 2024 21:05
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@raykallen raykallen raykallen approved these changes

@mdemoret-nv mdemoret-nv mdemoret-nv approved these changes

Assignees
No one assigned
Labels
improvement Improvement to existing functionality non-breaking Non-breaking change
Projects
Morpheus Boards
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@efajardo-nv @raykallen @mdemoret-nv