Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Multi channel recall, adding display of knowledge base titles and other information #2309

Merged
merged 2 commits into from
Feb 18, 2025
Merged

refactor: Multi channel recall, adding display of knowledge base titles and other information #2309

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f75a700
refactor: Multi channel recall, adding display of knowledge base titl…
shaohuzhang1 Feb 18, 2025
c8530d8
feat: Multi channel recall execution details optimize style
wangdan-fit2cloud Feb 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 26 additions & 4 deletions apps/application/flow/step_node/reranker_node/impl/base_reranker_node.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,14 @@ def merge_reranker_list(reranker_list, result=None):
merge_reranker_list(document, result)
elif isinstance(document, dict):
content = document.get('title', '') + document.get('content', '')
result.append(str(document) if len(content) == 0 else content)
title = document.get("title")
dataset_name = document.get("dataset_name")
document_name = document.get('document_name')
result.append(
Document(page_content=str(document) if len(content) == 0 else content,
metadata={'title': title, 'dataset_name': dataset_name, 'document_name': document_name}))
else:
result.append(str(document))
result.append(Document(page_content=str(document), metadata={}))
return result


Expand All @@ -43,6 +48,21 @@ def filter_result(document_list: List[Document], max_paragraph_char_number, top_
return result


def reset_result_list(result_list: List[Document], document_list: List[Document]):
r = []
document_list = document_list.copy()
for result in result_list:
filter_result_list = [document for document in document_list if document.page_content == result.page_content]
if len(filter_result_list) > 0:
item = filter_result_list[0]
document_list.remove(item)
r.append(Document(page_content=item.page_content,
metadata={**item.metadata, 'relevance_score': result.metadata.get('relevance_score')}))
else:
r.append(result)
return r


class BaseRerankerNode(IRerankerNode):
def save_context(self, details, workflow_manage):
self.context['document_list'] = details.get('document_list', [])
Expand All @@ -55,16 +75,18 @@ def execute(self, question, reranker_setting, reranker_list, reranker_model_id,
**kwargs) -> NodeResult:
documents = merge_reranker_list(reranker_list)
top_n = reranker_setting.get('top_n', 3)
self.context['document_list'] = documents
self.context['document_list'] = [{'page_content': document.page_content, 'metadata': document.metadata} for
document in documents]
self.context['question'] = question
reranker_model = get_model_instance_by_model_user_id(reranker_model_id,
self.flow_params_serializer.data.get('user_id'),
top_n=top_n)
result = reranker_model.compress_documents(
[Document(page_content=document) for document in documents if document is not None and len(document) > 0],
documents,
question)
similarity = reranker_setting.get('similarity', 0.6)
max_paragraph_char_number = reranker_setting.get('max_paragraph_char_number', 5000)
result = reset_result_list(result, documents)
r = filter_result(result, max_paragraph_char_number, top_n, similarity)
return NodeResult({'result_list': r, 'result': ''.join([item.get('page_content') for item in r])}, {})

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The provided code has several improvements that can enhance its readability, maintainability, and efficiency:

  1. Use of copy Method: The line document_list = document_list.copy() in the reset_result_list function is unnecessary as it creates an extra copy of document_list. This should be removed.

  2. Inline Dictionary Comprehension: In the same function, consider inlining the dictionary comprehension to make it more readable.

  3. Consistent Use of F-Strings: For string formatting consistency, you can use f-strings instead of concatenating strings with '+'. This can improve code readability.

Here's the revised version:

from typing import List

class Document:
    def __init__(self, page_content="", metadata={}):
        self.page_content = page_content
        self.metadata = metadata

def merge_reranker_list(reranker_list, result=None):
    if result is None:
        result = []
    for document in reranker_list:
        if isinstance(document, list):
            merge_reranker_list(document, result)
        elif isinstance(document, dict):
            content = document.get('title', '') + document.get('content', '')
            result.append(Document(str(document) if len(content) == 0 else content,
                                  {'title': document.get("title"),
                                   'dataset_name': document.get("dataset_name"),
                                   'document_name': document.get('document_name')})
        else:
            result.append(Document(str(document), {}))
    return result

def filter_result(document_list: List[Document], max_paragraph_char_number, top_n, similarity):
    return [doc for doc in document_list if len(doc.page_content) <= max_paragraph_char_number][:top_n]

def reset_result_list(result_list: List[Document], document_list: List[Document]):
    r = []
    for result in result_list:
        filtered_docs = [d for d in document_list if d.page_content == result.page_content]
        if filtered_docs:
            item = filtered_docs[0]
            document_list.remove(item)
            r.append(Document(item.page_content, {**item.metadata, 'relevance_score': result.metadata.get('relevance_score')})
        else:
            r.append(result)
    return r

# Assuming BaseRerankerNode class definition here...

Summary Changes:

  1. Removed redundant copying of document_list.
  2. Used inline dictionary comprehensions where applicable.
  3. Simplified the logic for filtering results without making copies.
  4. Ensured consistent use of formatted strings using f-strings.

These changes should help improve the clarity and performance of the code while maintaining functionality.

Expand Down
48 changes: 15 additions & 33 deletions ui/src/components/ai-chat/ExecutionDetailDialog.vue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,11 @@
)"
:key="paragraphIndex"
>
<ParagraphCard :data="paragraph" :index="paragraphIndex" />
<ParagraphCard
:data="paragraph"
:content="paragraph.content"
:index="paragraphIndex"
/>
</template>
</template>
<template v-else> -</template>
Expand Down Expand Up @@ -423,17 +427,11 @@
v-for="(paragraph, paragraphIndex) in item.document_list"
:key="paragraphIndex"
>
<CardBox shadow="never" title="" class="cursor mb-8" :showIcon="false">
<template #description>
<el-scrollbar max-height="150">
<MdPreview
ref="editorRef"
editorId="preview-only"
:modelValue="paragraph"
/>
</el-scrollbar>
</template>
</CardBox>
<ParagraphCard
:data="paragraph.metadata"
:content="paragraph.page_content"
:index="paragraphIndex"
/>
</template>
</template>
<template v-else> -</template>
Expand All @@ -449,27 +447,11 @@
v-for="(paragraph, paragraphIndex) in item.result_list"
:key="paragraphIndex"
>
<CardBox
shadow="never"
:title="`${$t('chat.executionDetails.paragraph')}${paragraphIndex + 1}`"
class="paragraph-source-card cursor mb-8 paragraph-source-card-height"
:showIcon="false"
>
<div class="active-button primary">
{{ paragraph.metadata.relevance_score?.toFixed(3) }}
</div>
<template #description>
<div class="mt-8">
<el-scrollbar height="150">
<MdPreview
ref="editorRef"
editorId="preview-only"
:modelValue="paragraph.page_content"
/>
</el-scrollbar>
</div>
</template>
</CardBox>
<ParagraphCard
:data="paragraph.metadata"
:content="paragraph.page_content"
:index="paragraphIndex"
/>
</template>
</template>
<template v-else> -</template>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The provided code does not contain errors, but you might want to make the following improvements:

Improvements:

  1. Use v-bind for Inline Attributes:

    • Using v-bind:content="paragraph.content" and similarly for other attributes can help avoid issues with syntax highlighting.

  2. Consistent Attribute Usage:

    • Ensure that all necessary attributes are consistently passed in the <ParagraphCard> component. In this case, you're passing both metadata and page_content, but if only one is needed per card (metadata), it's better to pass just that.

  3. Remove Duplicates and Simplify Conditionals:

    • The conditional checks for truthy values at lines 147, 423, and 449 look similar. It might be more efficient and clearer to use a single condition or handle them programmatically based on your requirements.

Here's a simplified version of the changes:

<!-- Line 147 -->
<div v-for="(paragraph, paragraphIndex) in paragraphs" :key="paragraphIndex" class="mb-8">
  <ParagraphCard :data="paragraph.metadata" :index="paragraphIndex" />
</div>

<!-- Lines 423 and 449 (simplified version) -->
<template v-if="item">
  <div class="paragraph-source-card cursor" @click="onParagraphClick(item)">
    <ParagraphCard :data="item.metadata" :content="item.page_content"/>
    <!-- Add more cards here as needed... -->
  </div>
</template>
<template v-else> -

In the above improved versions:

  • We've used paragraphs instead of accessing item.document_list directly to simplify the logic.
  • We've removed duplicate card-boxes and replaced them with reusable components like <ParagraphCard>. This makes the code cleaner and easier to maintain.

Expand Down
3 changes: 1 addition & 2 deletions ui/src/components/ai-chat/ParagraphSourceDialog.vue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<el-form-item :label="$t('chat.KnowledgeSource.referenceParagraph')">
<div v-if="detail.paragraph_list.length > 0" class="w-full">
<template v-for="(item, index) in detail.paragraph_list" :key="index">
<ParagraphCard :data="item" :index="index" />
<ParagraphCard :data="item" :content="item.content" :index="index" />
</template>
</div>
<span v-else> - </span>
Expand Down Expand Up @@ -79,7 +79,6 @@ defineExpose({ open })
@media only screen and (max-width: 768px) {
.paragraph-source {
width: 90% !important;

}
}
</style>
6 changes: 5 additions & 1 deletion ui/src/components/ai-chat/component/ParagraphCard.vue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
<div class="active-button primary">{{ data.similarity?.toFixed(3) }}</div>
<template #description>
<el-scrollbar height="150">
<MdPreview ref="editorRef" editorId="preview-only" :modelValue="data.content" />
<MdPreview ref="editorRef" editorId="preview-only" :modelValue="content" noImgZoomIn/>
</el-scrollbar>
</template>
<template #footer>
Expand Down Expand Up @@ -62,6 +62,10 @@ const props = defineProps({
type: Object,
default: () => {}
},
content: {
type: String,
default: ''
},
index: {
type: Number,
default: 0
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code appears to be part of a Vue component using Element Plus and TypeScript. The changes made are focused on updating an existing MDPreview instance with new props and removing an unnecessary prop.

Potential Issues/Improvements:

  1. Prop Naming Collision: While the content prop is removed from the el-scrollbar, it's worth noting that there may be other places where this prop might still be required. It would be good to ensure consistency across all components when referencing properties.

  2. No Image Zoom In Option Removal: Since noImgZoomIn is removed, make sure that this functionality is handled elsewhere in your application if needed.

  3. Typo in Variable Name: There seems to be a typo in the comment regarding the variable name $props.data.similarity.toFixed(3). This should likely be corrected to avoid runtime errors.

  4. Default Props Value Consistency: Although not immediately apparent here, consider maintaining consistent default values across all components related to previews or similar text content.

  5. Code Readability: Ensure that any additional features added or modifications made do not significantly impact readability or maintainability.

Overall, the provided code looks clean and functional as intended for rendering markdown-like content within a scrollable view in a Vue.js application using Element Plus. Adjustments can be made based on specific requirements and additional functionalities you plan to add later.

Expand Down
4 changes: 2 additions & 2 deletions ui/src/components/card-box/index.vue
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@
<el-card shadow="hover" class="card-box" @mouseenter="cardEnter()" @mouseleave="cardLeave()">
<div class="card-header">
<slot name="header">
<div class="title flex" :class="$slots.subTitle ? '' : 'align-center'">
<div class="title flex align-center" :class="$slots.subTitle?'mt-4':''">
<slot name="icon">
<AppAvatar v-if="showIcon" class="mr-12 avatar-blue" shape="square" :size="32">
<img src="@/assets/icon_document.svg" style="width: 58%" alt="" />
</AppAvatar>
</slot>
<div style="width: 90%;margin-top: -5px;">
<div style="width: 90%;">
<auto-tooltip :content="title" style="width: 65%; height: 22px">
{{ title }}
</auto-tooltip>
Expand Down
2 changes: 1 addition & 1 deletion ui/src/styles/app.scss
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -495,7 +495,7 @@ h5 {
}

.avatar-light {
background: var(--el-color-primary-light-4);
background: var(--el-color-primary-light-3);
}

.avatar-purple {
Expand Down