Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: deep dive alerts #851

Closed

Conversation

matthisholleville
Copy link
Contributor

@matthisholleville matthisholleville commented Jan 8, 2024

Closes #795

📑 Description

Disclaimer : This PR is a first implementation and should be consolidated

It's deeply reworks our interaction with the AI. Until now, we had an AI agent (which we could call "analyzer"). Its goal is to provide an explanation for an error sample. This agent is still used for the "classic" explain mode.

For "deep dive alert" analysis, it was necessary to transform this model to switch to a multi-agent autonomous AI system.

This system consists of 3 AI agents:

  1. router: its goal is to identify the appropriate analyzers to activate based on the provided alert (k8sgpt analyze --alert ...).
  2. evaluator: once the analysis has been performed, this agent's objective is to filter errors related to the provided alert. This agent helps refine errors that may be related to the alert.
  3. analyzer: it's the historical analyzer responsible for finding a solution to a problem.

A new pkg called agent has been created to host the code for these agents.

full-demo-deep-dive.mov

✅ Checks

  • My pull request adheres to the code style of this project
  • My code requires changes to the documentation
  • I have updated the documentation as required
  • All the tests have passed
  • Add Copyright in new files

ℹ Additional Information

Other improvements:

  • A loader has been added during analysis to enhance the user experience in CLI mode ( we should not display the loader if json format provided )
  • A description has been added to each of the analyzers to provide context to the "router" agent about the purpose of the analyzer. removed due to frequent maxToken hit

This PR is a first implementation and should be consolidated with the following elements:

  • Modification of k8sgpt-schemas to add the concept of alerts.
  • The prompts need to be tested with different alerts.
  • The documentation needs to be updated.
  • Test with other AI Provider ( tested with OpenAI )
  • Do not display the loader if the json format is provided.

With this mode, we make 2 additional AI API calls per command (router and evaluator agent).


Analysis flow with error

image

Signed-off-by: Matthis Holleville <matthish29@gmail.com>
…s/integrations

Signed-off-by: Matthis Holleville <matthish29@gmail.com>
Signed-off-by: Matthis Holleville <matthish29@gmail.com>
@bwplotka bwplotka mentioned this pull request Jan 9, 2024
4 tasks
cmd/analyze/analyze.go Show resolved Hide resolved
@@ -88,6 +122,8 @@ func init() {
AnalyzeCmd.Flags().BoolVarP(&nocache, "no-cache", "c", false, "Do not use cached data")
// anonymize flag
AnalyzeCmd.Flags().BoolVarP(&anonymize, "anonymize", "a", false, "Anonymize data before sending it to the AI backend. This flag masks sensitive data, such as Kubernetes object names and labels, by replacing it with a key. However, please note that this flag does not currently apply to events.")
// alert flag
AnalyzeCmd.Flags().StringVarP(&alert, "alert", "", "", "Alert to be analyzed")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's offer a shorthand flag also, it doesn't need to be "-a"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about "l" ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's fine, might be easier to make it a less ambiguous char like g or y

pkg/agent/analyzer.go Show resolved Hide resolved
@AlexsJones
Copy link
Member

What makes this OpenAI specific? I expected to see us use OpenAI functions, but it looks like we're just doing some smart prompt manipulation instead.

@matthisholleville
Copy link
Contributor Author

We had the choice between using OpenAI's assistant mode and being specific to OpenAI, or using a multi-agent system that we maintain ourselves. This is the choice I made so that this mode is available for all AI backends.

@AlexsJones
Copy link
Member

We had the choice between using OpenAI's assistant mode and being specific to OpenAI, or using a multi-agent system that we maintain ourselves. This is the choice I made so that this mode is available for all AI backends.

Okay that's even better. I am looking for an opportunity to re-use some of your work in the interactive loop I am making

Signed-off-by: Matthis Holleville <matthish29@gmail.com>
Signed-off-by: Matthis Holleville <matthish29@gmail.com>
@matthisholleville
Copy link
Contributor Author

If you need help or suggestions for integrating multi-agent functionality, reach out to me.

I can finalize the few remaining jobs quickly to have a 'ready for review' pull request.

Signed-off-by: Matthis Holleville <matthish29@gmail.com>
Signed-off-by: Matthis Holleville <matthish29@gmail.com>
Signed-off-by: Matthis Holleville <matthish29@gmail.com>
Signed-off-by: Matthis Holleville <matthish29@gmail.com>
… in the prompt and providing it with more error context.

Signed-off-by: Matthis Holleville <matthish29@gmail.com>
@AlexsJones
Copy link
Member

Sorry I lost this thread @matthisholleville - I think my primary reservation was was originally around it being bound to OpenAI functionality; I think you subsequently disproved that. Shall we get this through another review? The schema has just been approved.

@matthisholleville
Copy link
Contributor Author

@AlexsJones I'm diving back into it this week and I'll see about getting it ready for review as quickly as possible.

@AlexsJones AlexsJones closed this May 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

[Feature]: Deep dive alerts
2 participants