How expand Kernel Memory Output #927

peopleworks · 2024-12-04T15:42:46Z

peopleworks
Dec 4, 2024

Hi

Everything is working fine, I get the right answer, fast response, but the answer is cut off at one point and does not give me completeness even though the part it presents is correct and in the order that I requested it.

`
public async Task AskInsightsAsync()
{

var question = @"
Data Insights Extraction Prompt

Objective: Perform an in-depth analysis of the provided JSON file, uncovering meaningful insights, patterns, and actionable intelligence.

Data Exploration Steps:

1. Visualization Recommendations
- Recommend the most appropriate visualization types for key insights
- Using Mermaid diagrams suggest potential charts, graphs, or diagrams that would effectively communicate the data's story. Ensure the diagram is clear and concise, avoiding over-complication or information overload

2. Comprehensive Data Profiling
- Calculate and report:
  * Total number of records/entries
  * Distribution of data across different keys
  * Unique value counts for categorical variables
  * Statistical summary for numerical variables (mean, median, min, max, standard deviation)

3. Key Insights Extraction
- Identify the most significant patterns and trends in the data
- Highlight:
  * Top N values for important categorical variables
  * Unusual or outlier data points
  * Correlations between different attributes
  * Potential hidden relationships in the data

4. Advanced Analysis
- Perform contextual analysis based on the nature of the data
- Develop hypotheses about potential underlying patterns
- Suggest potential business or research implications of the findings


5. Data Quality Assessment
- Evaluate data completeness
- Identify missing values or inconsistencies
- Assess the reliability and potential biases in the dataset

Deliverables:
- Comprehensive written report of findings
- Summary of key insights
- Recommendations for further investigation
- Potential data-driven action items

Additional Instructions:
- Provide clear, concise explanations
- Use technical yet accessible language
- Back up observations with specific data references
- Highlight both obvious and non-obvious insights

Formatting Preferences:
- Always respond in proper HTML format, but do not include <html>, <head>, or <body> tags.
- Include headers and subheaders
- Utilize bullet points and numbered lists for readability
- Provide code snippets or pseudo-code if relevant

Please process the JSON file with these guidelines, demonstrating deep analytical thinking and attention to detail.
";

var answer = await memoryConnector.AskAsync(question, filter: MemoryFilters.ByDocument("json01"));

return answer.Result;

}
`

Thank you in advance

Answered by dluc

Dec 6, 2024

Hi @peopleworks, it sounds like you're looking for more of a "document analysis" approach, which aligns better with agentic AI than with standard RAG. RAG typically treats queries as simple strings and finds similar strings. What you're describing is more like a multi-step plan that an agent can execute, possibly using RAG as part of the process. You might consider exploring AI agent frameworks to achieve what you have in mind.

View full answer

dluc · 2024-12-06T07:19:25Z

dluc
Dec 6, 2024
Maintainer

Hi @peopleworks, it sounds like you're looking for more of a "document analysis" approach, which aligns better with agentic AI than with standard RAG. RAG typically treats queries as simple strings and finds similar strings. What you're describing is more like a multi-step plan that an agent can execute, possibly using RAG as part of the process. You might consider exploring AI agent frameworks to achieve what you have in mind.

0 replies

peopleworks · 2024-12-10T04:06:04Z

peopleworks
Dec 10, 2024
Author

Hi @dluc

Tested is a Kernel Memory problem, I use only Semantic Kernel + ITextEmbeddingGenerationService + tokenizer and works perfect

2024-12-09_22-52-18.SKTK.mp4

Maybe another better way that I can persist the file and use from a Database.

Thank you
Pedro

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How expand Kernel Memory Output #927

{{title}}

Replies: 2 comments

{{title}}

{{title}}

Select a reply

How expand Kernel Memory Output #927

peopleworks Dec 4, 2024

Replies: 2 comments

dluc Dec 6, 2024 Maintainer

peopleworks Dec 10, 2024 Author

peopleworks
Dec 4, 2024

dluc
Dec 6, 2024
Maintainer

peopleworks
Dec 10, 2024
Author