PDF and DOCX support in Write File - Feature Improvement, close #548 #1125

Arkajit-Datta · 2023-08-25T12:02:30Z

Description

The Write File Tool can now support writing the results and discussions in a PDF or a DOCX file, with proper formatting and images embedded.

Related Issues

#548

Solution and Design

Libraries called pdfkit and htmldocx are being leveraged to do the writing process.
The content initially used to be stored in a Txt file using the write tool is formatted and styled through an HTML file using an LLM.
Images generated by the agent are extracted and embedded in the same HTML file.
This HTML file is now stored in various formats like PDF & DOCX.

Test Plan

Asking the Agent to store the results in a PDF or DOCX file.
Navigate to the workspace folder to find the generated PDF or DOCX.

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Docs update

Checklist

My pull request is atomic and focuses on a single change.
I have read the contributing guide and my code conforms to the guidelines.
I have documented my changes clearly and comprehensively.
I have added the required tests.

…ular tools

… generated by the agent during the run

… are fetched inorder to be embedded in the respective file type.

luciferlinx101 · 2023-08-25T12:07:41Z

superagi/models/agent_execution_feed.py

@@ -47,12 +47,14 @@ def get_last_tool_response(cls, session: Session, agent_execution_id: int, tool_
            AgentExecutionFeed.role == "system").order_by(AgentExecutionFeed.created_at.desc()).all()

        for agent_execution_feed in agent_execution_feeds:


revert unnecessary changes

Reverted the changes.

superagi/resource_manager/file_manager.py

luciferlinx101 · 2023-08-25T12:23:28Z

superagi/tools/tool_response_query_manager.py

@@ -11,7 +11,7 @@ def __init__(self, session: Session, agent_execution_id: int,memory:VectorStore)



dont touch other files

So, I created a functions here which was not needed after a change in dev. I removed the function while merging

luciferlinx101 · 2023-08-25T12:34:05Z

superagi/resource_manager/file_manager.py

-            final_path = ResourceHelper.get_resource_path(file_name)
+
+        logger.info(f"{file_name} - File written successfully")
+


why write_file is returning final_path?

It was previously being used in some other function, which was removed hence removing this!

luciferlinx101 · 2023-08-25T12:39:32Z

superagi/resource_manager/file_manager.py

+        }
+
+        if file_type in file_type_handlers:
+            return file_type_handlers[file_type](file_name, file_path, content)


where is this return needed?

Removed the return!

luciferlinx101 · 2023-08-25T12:40:46Z

superagi/resource_manager/file_manager.py

@@ -45,7 +56,7 @@ def write_to_s3(self, file_name, final_path):
                s3_helper = S3Helper()
                s3_helper.upload_file(img, path=resource.path)

-    def write_file(self, file_name: str, content):
+    def write_file(self, file_name: str, content, return_file_path: bool = False):
        if self.agent_id is not None:


Dont pass return_file_path, here in write file path param

It was aiding another function, seems like obsolete now, removing it!

luciferlinx101 · 2023-08-25T12:47:25Z

superagi/tools/file/write_file.py

+            if file_extension.lower() in image_extensions:
+                image_paths.append(path)
+        return image_paths
+


It is better to remove this _get_file_path and directly call ResourceHelper()... without using any temporary variable like it is being used in other places in the code

That sounds fair! Removing the function

codecov · 2023-08-26T10:38:29Z

Codecov Report

Patch coverage: 54.41% and project coverage change: -0.10% ⚠️

Comparison is base (6817934) 58.88% compared to head (6275fba) 58.78%.
Report is 2 commits behind head on dev.

Additional details and impacted files

@@            Coverage Diff             @@
##              dev    #1125      +/-   ##
==========================================
- Coverage   58.88%   58.78%   -0.10%     
==========================================
  Files         218      221       +3     
  Lines       10510    10769     +259     
  Branches     1118     1147      +29     
==========================================
+ Hits         6189     6331     +142     
- Misses       4015     4125     +110     
- Partials      306      313       +7

Files Changed	Coverage Δ
...uperagi/resource_manager/llama_document_summary.py	`28.12% <ø> (-6.17%)`	⬇️
superagi/tools/github/github_toolkit.py	`0.00% <0.00%> (ø)`
superagi/tools/file/write_file.py	`41.02% <28.12%> (-53.10%)`	⬇️
superagi/resource_manager/file_manager.py	`41.73% <31.81%> (-8.87%)`	⬇️
superagi/controllers/organisation.py	`56.25% <33.33%> (+2.59%)`	⬆️
superagi/exceptions/file_exceptions.py	`50.00% <50.00%> (ø)`
superagi/helper/github_helper.py	`58.82% <55.31%> (-1.34%)`	⬇️
superagi/tools/github/review_pull_request.py	`72.15% <72.15%> (ø)`
superagi/llms/llm_model_factory.py	`47.50% <90.00%> (+17.50%)`	⬆️
superagi/tools/github/fetch_pull_request.py	`100.00% <100.00%> (ø)`

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Arkajit-Datta · 2023-08-29T07:38:16Z

Hey @luciferlinx101 please review the PR whenever free thanx!

superagi/exceptions/file_errors.py

Dockerfile

Arkajit-Datta · 2023-08-30T03:10:17Z

Sure @luciferlinx101 I'll do these changes as soon as possible

luciferlinx101 · 2023-08-31T11:41:11Z

Also, add required comments/documentation as maintained in the code for all methods and classes, mark the PR checklist for the same.

Arkajit-Datta · 2023-09-04T11:46:35Z

@luciferlinx101 i have added the unit tests and the other review comments, Please have a look thanx

superagi/models/agent_execution_feed.py

superagi/tools/tool_response_query_manager.py

superagi/tools/image_generation/stable_diffusion_image_gen.py

#548 (#1125)" This reverts commit 26f6a1d.

#548 (#1125)" (#1202) This reverts commit 26f6a1d.

…ent, close #548 (#1125)" (#1202)" This reverts commit 3153def.

* fix * fixing the toolkit config in iteration workflow * List file s3 fix (#1076) * List File S3 * Unit Test Added --------- Co-authored-by: Taran <97586318+Tarraann@users.noreply.github.com> * workflow changes * minor seed file fix * frontend fixes (#1079) * fixed api_bug (#1080) * add one condition (#1082) * api fix (#1087) * Tools error fix (#1093) * webhooks frontend + api calls complete almost * Tool-LTM(Updated) (#1039) * Toolkit configuration fix (#1102) * webhooks compplete frontend * schedule agent fix (#1104) Co-authored-by: Rounak Bhatia <rounak@contlo.com> * Models superagi (#936) Models Superagi * Models superagi (#1108) Bug Fixes * Changes for no receiver address * made changes to github helper * Models superagi (#1112) * Models superagi (#1117) * Models fixes (#1118) * Models Frontend Changes * Models Frontend Changes * Models Frontend Changes * \n bug resolved (#1122) * PDF and DOCX support in Write File - Feature Improvement, close #548 (#928) * Added functions to write various file types and a file handler too * FileManager updated to handle and save HTMLs. * adding PDF + DOCX support to save images * Added Wkhtmltopdf package installation run commands in docker * Added get_all_responses feature for extractng the response for particular tools * Added Image embedding feature, this will extract and embed the images generated by the agent during the run * renaming functions and refactoring * renaming functions and refactoring * Update Dockerfile * removing unsused classmethods * Finding generated images and attached files in the write tool. Images are fetched inorder to be embedded in the respective file type. * Adding the filename and paths to the Resource Manager S3 storage * Code Cleanup * added logger: Fix for the failing TEST --------- Co-authored-by: Fluder-Paradyne <121793617+Fluder-Paradyne@users.noreply.github.com> * Revert "PDF and DOCX support in Write File - Feature Improvement, close #548 (#928)" (#1124) This reverts commit 8b01357. * expose port * latest safetensors breaking in macs (#1134) * Changes in save template (#1120) * fix * changes in save template * changes in save template * bug fixes * changes in save template * changes in save template --------- Co-authored-by: Rounak Bhatia <rounak@contlo.com> Co-authored-by: namansleeps <mandhani12@gmail.com> * Main to dev sync (#1139) * Models fixes (#1126) * Models Frontend Changes * Models Frontend Changes * Models Frontend Changes * Backend Compatibility for New/Existing users on local * DEV api key requirements * removing print statements * removing print statements * removing print statements * removing print statements * backend compatibility * backend compatibility * backend compatibility * added filters in the webhooks * fix * added filters in the webhooks * Models fixes (#1145) Fixes related to Models Feature * Jira Bug Fix * Jira Bug Fix 2.0 * Jira Bug Fix 3.0 * added filters in the webhooks * Models fixes (#1147) Model Feature Fixes * Bug fix model redirection (#1148) Bug Fix - Model URL Redirection * added tool config for dalle * removed model dependency on dalle tool * Remove hardcoded creds * fixed env error * removed refactoring from main * removed refactoring * removed refactoring * handled error * stop agent from executing if model is not found (#1156) * entity details (#1158) * Metric frontend (#1152) * added filters in webhooks * added filters in webhooks * minor changes * webhooks complete * minor changes for PR * minor changes for PR * Publish agent template to marketplace (#1106) * publish agent to marketplace --------- Co-authored-by: Rounak Bhatia <rounak@contlo.com> Co-authored-by: namansleeps <mandhani12@gmail.com> * added filters in webhooks * added filters in webhooks * resolving conflicts * added filters in the webhooks * lint issue fixed * bug fix of prev PR * fix for new run and edit agent * error handling * added filters in webhooks * fix for knowledge search tool * Docker digitalocean deployment * changed branch name * added filters in the webhooks * changes * removed region * added button * change in branch * added filters in the webhooks * Update conftest.py * Added filters in the webhooks (#1140) * webhooks frontend + web hooks with filters --------- Co-authored-by: namansleeps <mandhani12@gmail.com> Co-authored-by: Fluder-Paradyne <121793617+Fluder-Paradyne@users.noreply.github.com> * Models calls logs dev (#1174) Call logs organisation level bug fix * models scroll fix, format of log timestamp fix, adding of loader to models, toolkit metrics dropdown bug fixed, publish agent dropdown bug (#1171) * Update app.yaml (#1179) * fixes related to webhooks * fixes for webhooks * Fixes for webhooks (#1181) * fixes for webhooks --------- Co-authored-by: namansleeps <mandhani12@gmail.com> Co-authored-by: Fluder-Paradyne <121793617+Fluder-Paradyne@users.noreply.github.com> * bugs by qa (#1178) * Fix for schedule agent (#1184) Co-authored-by: Tarraann <jot.taran15522@gmail.com> * Entity fix (#1185) * fixes for webhooks * fixes for webhooks * fix added for index state (#1188) * fix added for index state * Update KnowledgeTemplate.js --------- Co-authored-by: Tarraann <jot.taran15522@gmail.com> * API bug fixes for SDK (#1189) * fix api's for sdk * removed unused imports --------- Co-authored-by: jagtarcontlo <123375045+jagtarcontlo@users.noreply.github.com> * Main to dev sync v12 (#1193) sync back to dev ------ Co-authored-by: Taran <97586318+Tarraann@users.noreply.github.com> Co-authored-by: TransformerOptimus <muknrq@gmail.com> Co-authored-by: Fluder-Paradyne <121793617+Fluder-Paradyne@users.noreply.github.com> Co-authored-by: Maverick-F35 <138012351+Maverick-F35@users.noreply.github.com> Co-authored-by: BoundlessAsura <122777244+boundless-asura@users.noreply.github.com> Co-authored-by: Akshat Jain <92881074+Akki-jain@users.noreply.github.com> Co-authored-by: sayan1101 <139119661+sayan1101@users.noreply.github.com> Co-authored-by: Rounak Bhatia <f20201807@goa.bits-pilani.ac.in> Co-authored-by: Rounak Bhatia <rounak@contlo.com> Co-authored-by: Kalki <97698934+jedan2506@users.noreply.github.com> Co-authored-by: Tarraann <jot.taran15522@gmail.com> Co-authored-by: rakesh-krishna-a-s <akrishna@contlo.com> Co-authored-by: Captain Levi <123375045+CaptainLevi0007@users.noreply.github.com> Co-authored-by: andrew-kelly-neutralaiz <128111428+andrew-kelly-neutralaiz@users.noreply.github.com> Co-authored-by: James Wakelim <james.wakelim@neutralaiz.com> * added button * GitHub pull request tools (#1190) * adding github review tools * cleanup and adding code review prompt * fixing comments * PDF and DOCX support in Write File - Feature Improvement, close #548 (#1125) Co-authored-by: Fluder-Paradyne <121793617+Fluder-Paradyne@users.noreply.github.com> Co-authored-by: Abhijeet <129729795+luciferlinx101@users.noreply.github.com> * minor documentation fix * Design bugs (#1199) * fetching token limit from db * Revert "PDF and DOCX support in Write File - Feature Improvement, close #548 (#1125)" (#1202) This reverts commit 26f6a1d. * Unit Test Fix (#1203) * adding of docs and and discord link correction (#1205) * openai error handling * error_handling * api call only when agent is running * Feature : Wait block for agent workflow (#1186) Agent Wait Block Step * minor changes (#1213) Co-authored-by: Jagtar Saggu <jagtarsaggu@Jagtars-MacBook-Pro.local> * error handling * error handling * error handling * error handling * error handling * fix * fix * fix * error handling * models changes (#1207) Model related frontend changes. * error handling * models marketplace changes (#1219) Co-authored-by: Abhijeet <129729795+luciferlinx101@users.noreply.github.com> * minor changes * error handling * error handling * removing single qoutes (#1224) Co-authored-by: namansleeps <mandhani12@gmail.com> * apm changes (#1222) APM Bug Fixes * list tool fix * list tool fix * PR CHANGES * entity fix for dev (#1230) Entity Fix for Dev * frontend changes (#1231) * read_tool_fix * fix * waiting block frontend (#1233) Waiting Block Changes and Frontend Addition * Dev Fixes (#1242) * read tool fix * Maintaining dev (#1244) Dev Fix * added logs (#1246) * error_handling fix (#1247) Error handling fix. * Feature first login src (#1241) Adding source in user database for analytics. * apollo NoneType bug fix (#1238) Bug Fix * Mixpanel integration (#1256) Mix Panel * Models Marketplace bug fix for dev (#1266) * Fix 1257 dev (#1269) Bug Fix * add cache layer (#1275) Added package caching for github actions workflow * Fix api dev (#1283) * save other config to agent_execution config * add config * mixpanel changes (#1285) * revert README.MD * rename error_handling.py to error_handler.py (#1287) --------- Co-authored-by: Rounak Bhatia <rounak@contlo.com> Co-authored-by: TransformerOptimus <muknrq@gmail.com> Co-authored-by: Abhijeet <129729795+luciferlinx101@users.noreply.github.com> Co-authored-by: Taran <97586318+Tarraann@users.noreply.github.com> Co-authored-by: Maverick-F35 <138012351+Maverick-F35@users.noreply.github.com> Co-authored-by: Rounak Bhatia <f20201807@goa.bits-pilani.ac.in> Co-authored-by: namansleeps <mandhani12@gmail.com> Co-authored-by: Aditya Sharma <138581531+AdityaSharma13064@users.noreply.github.com> Co-authored-by: Sayan Samanta <139119661+sayan1101@users.noreply.github.com> Co-authored-by: Kalki <97698934+jedan2506@users.noreply.github.com> Co-authored-by: Tarraann <jot.taran15522@gmail.com> Co-authored-by: Arkajit Datta <61142632+Arkajit-Datta@users.noreply.github.com> Co-authored-by: jagtarcontlo <123375045+jagtarcontlo@users.noreply.github.com> Co-authored-by: Captain Levi <123375045+CaptainLevi0007@users.noreply.github.com> Co-authored-by: namansleeps <122260931+namansleeps@users.noreply.github.com> Co-authored-by: I’m <133493246+TransformerOptimus@users.noreply.github.com> Co-authored-by: Jagtar Saggu <jagtarsaggu@Jagtars-MacBook-Pro.local>

* fix * fixing the toolkit config in iteration workflow * List file s3 fix (TransformerOptimus#1076) * List File S3 * Unit Test Added --------- Co-authored-by: Taran <97586318+Tarraann@users.noreply.github.com> * workflow changes * minor seed file fix * frontend fixes (TransformerOptimus#1079) * fixed api_bug (TransformerOptimus#1080) * add one condition (TransformerOptimus#1082) * api fix (TransformerOptimus#1087) * Tools error fix (TransformerOptimus#1093) * webhooks frontend + api calls complete almost * Tool-LTM(Updated) (TransformerOptimus#1039) * Toolkit configuration fix (TransformerOptimus#1102) * webhooks compplete frontend * schedule agent fix (TransformerOptimus#1104) Co-authored-by: Rounak Bhatia <rounak@contlo.com> * Models superagi (TransformerOptimus#936) Models Superagi * Models superagi (TransformerOptimus#1108) Bug Fixes * Changes for no receiver address * made changes to github helper * Models superagi (TransformerOptimus#1112) * Models superagi (TransformerOptimus#1117) * Models fixes (TransformerOptimus#1118) * Models Frontend Changes * Models Frontend Changes * Models Frontend Changes * \n bug resolved (TransformerOptimus#1122) * PDF and DOCX support in Write File - Feature Improvement, close TransformerOptimus#548 (TransformerOptimus#928) * Added functions to write various file types and a file handler too * FileManager updated to handle and save HTMLs. * adding PDF + DOCX support to save images * Added Wkhtmltopdf package installation run commands in docker * Added get_all_responses feature for extractng the response for particular tools * Added Image embedding feature, this will extract and embed the images generated by the agent during the run * renaming functions and refactoring * renaming functions and refactoring * Update Dockerfile * removing unsused classmethods * Finding generated images and attached files in the write tool. Images are fetched inorder to be embedded in the respective file type. * Adding the filename and paths to the Resource Manager S3 storage * Code Cleanup * added logger: Fix for the failing TEST --------- Co-authored-by: Fluder-Paradyne <121793617+Fluder-Paradyne@users.noreply.github.com> * Revert "PDF and DOCX support in Write File - Feature Improvement, close TransformerOptimus#548 (TransformerOptimus#928)" (TransformerOptimus#1124) This reverts commit 8b01357. * expose port * latest safetensors breaking in macs (TransformerOptimus#1134) * Changes in save template (TransformerOptimus#1120) * fix * changes in save template * changes in save template * bug fixes * changes in save template * changes in save template --------- Co-authored-by: Rounak Bhatia <rounak@contlo.com> Co-authored-by: namansleeps <mandhani12@gmail.com> * Main to dev sync (TransformerOptimus#1139) * Models fixes (TransformerOptimus#1126) * Models Frontend Changes * Models Frontend Changes * Models Frontend Changes * Backend Compatibility for New/Existing users on local * DEV api key requirements * removing print statements * removing print statements * removing print statements * removing print statements * backend compatibility * backend compatibility * backend compatibility * added filters in the webhooks * fix * added filters in the webhooks * Models fixes (TransformerOptimus#1145) Fixes related to Models Feature * Jira Bug Fix * Jira Bug Fix 2.0 * Jira Bug Fix 3.0 * added filters in the webhooks * Models fixes (TransformerOptimus#1147) Model Feature Fixes * Bug fix model redirection (TransformerOptimus#1148) Bug Fix - Model URL Redirection * added tool config for dalle * removed model dependency on dalle tool * Remove hardcoded creds * fixed env error * removed refactoring from main * removed refactoring * removed refactoring * handled error * stop agent from executing if model is not found (TransformerOptimus#1156) * entity details (TransformerOptimus#1158) * Metric frontend (TransformerOptimus#1152) * added filters in webhooks * added filters in webhooks * minor changes * webhooks complete * minor changes for PR * minor changes for PR * Publish agent template to marketplace (TransformerOptimus#1106) * publish agent to marketplace --------- Co-authored-by: Rounak Bhatia <rounak@contlo.com> Co-authored-by: namansleeps <mandhani12@gmail.com> * added filters in webhooks * added filters in webhooks * resolving conflicts * added filters in the webhooks * lint issue fixed * bug fix of prev PR * fix for new run and edit agent * error handling * added filters in webhooks * fix for knowledge search tool * Docker digitalocean deployment * changed branch name * added filters in the webhooks * changes * removed region * added button * change in branch * added filters in the webhooks * Update conftest.py * Added filters in the webhooks (TransformerOptimus#1140) * webhooks frontend + web hooks with filters --------- Co-authored-by: namansleeps <mandhani12@gmail.com> Co-authored-by: Fluder-Paradyne <121793617+Fluder-Paradyne@users.noreply.github.com> * Models calls logs dev (TransformerOptimus#1174) Call logs organisation level bug fix * models scroll fix, format of log timestamp fix, adding of loader to models, toolkit metrics dropdown bug fixed, publish agent dropdown bug (TransformerOptimus#1171) * Update app.yaml (TransformerOptimus#1179) * fixes related to webhooks * fixes for webhooks * Fixes for webhooks (TransformerOptimus#1181) * fixes for webhooks --------- Co-authored-by: namansleeps <mandhani12@gmail.com> Co-authored-by: Fluder-Paradyne <121793617+Fluder-Paradyne@users.noreply.github.com> * bugs by qa (TransformerOptimus#1178) * Fix for schedule agent (TransformerOptimus#1184) Co-authored-by: Tarraann <jot.taran15522@gmail.com> * Entity fix (TransformerOptimus#1185) * fixes for webhooks * fixes for webhooks * fix added for index state (TransformerOptimus#1188) * fix added for index state * Update KnowledgeTemplate.js --------- Co-authored-by: Tarraann <jot.taran15522@gmail.com> * API bug fixes for SDK (TransformerOptimus#1189) * fix api's for sdk * removed unused imports --------- Co-authored-by: jagtarcontlo <123375045+jagtarcontlo@users.noreply.github.com> * Main to dev sync v12 (TransformerOptimus#1193) sync back to dev ------ Co-authored-by: Taran <97586318+Tarraann@users.noreply.github.com> Co-authored-by: TransformerOptimus <muknrq@gmail.com> Co-authored-by: Fluder-Paradyne <121793617+Fluder-Paradyne@users.noreply.github.com> Co-authored-by: Maverick-F35 <138012351+Maverick-F35@users.noreply.github.com> Co-authored-by: BoundlessAsura <122777244+boundless-asura@users.noreply.github.com> Co-authored-by: Akshat Jain <92881074+Akki-jain@users.noreply.github.com> Co-authored-by: sayan1101 <139119661+sayan1101@users.noreply.github.com> Co-authored-by: Rounak Bhatia <f20201807@goa.bits-pilani.ac.in> Co-authored-by: Rounak Bhatia <rounak@contlo.com> Co-authored-by: Kalki <97698934+jedan2506@users.noreply.github.com> Co-authored-by: Tarraann <jot.taran15522@gmail.com> Co-authored-by: rakesh-krishna-a-s <akrishna@contlo.com> Co-authored-by: Captain Levi <123375045+CaptainLevi0007@users.noreply.github.com> Co-authored-by: andrew-kelly-neutralaiz <128111428+andrew-kelly-neutralaiz@users.noreply.github.com> Co-authored-by: James Wakelim <james.wakelim@neutralaiz.com> * added button * GitHub pull request tools (TransformerOptimus#1190) * adding github review tools * cleanup and adding code review prompt * fixing comments * PDF and DOCX support in Write File - Feature Improvement, close TransformerOptimus#548 (TransformerOptimus#1125) Co-authored-by: Fluder-Paradyne <121793617+Fluder-Paradyne@users.noreply.github.com> Co-authored-by: Abhijeet <129729795+luciferlinx101@users.noreply.github.com> * minor documentation fix * Design bugs (TransformerOptimus#1199) * fetching token limit from db * Revert "PDF and DOCX support in Write File - Feature Improvement, close TransformerOptimus#548 (TransformerOptimus#1125)" (TransformerOptimus#1202) This reverts commit 26f6a1d. * Unit Test Fix (TransformerOptimus#1203) * adding of docs and and discord link correction (TransformerOptimus#1205) * openai error handling * error_handling * api call only when agent is running * Feature : Wait block for agent workflow (TransformerOptimus#1186) Agent Wait Block Step * minor changes (TransformerOptimus#1213) Co-authored-by: Jagtar Saggu <jagtarsaggu@Jagtars-MacBook-Pro.local> * error handling * error handling * error handling * error handling * error handling * fix * fix * fix * error handling * models changes (TransformerOptimus#1207) Model related frontend changes. * error handling * models marketplace changes (TransformerOptimus#1219) Co-authored-by: Abhijeet <129729795+luciferlinx101@users.noreply.github.com> * minor changes * error handling * error handling * removing single qoutes (TransformerOptimus#1224) Co-authored-by: namansleeps <mandhani12@gmail.com> * apm changes (TransformerOptimus#1222) APM Bug Fixes * list tool fix * list tool fix * PR CHANGES * entity fix for dev (TransformerOptimus#1230) Entity Fix for Dev * frontend changes (TransformerOptimus#1231) * read_tool_fix * fix * waiting block frontend (TransformerOptimus#1233) Waiting Block Changes and Frontend Addition * Dev Fixes (TransformerOptimus#1242) * read tool fix * Maintaining dev (TransformerOptimus#1244) Dev Fix * added logs (TransformerOptimus#1246) * error_handling fix (TransformerOptimus#1247) Error handling fix. * Feature first login src (TransformerOptimus#1241) Adding source in user database for analytics. * apollo NoneType bug fix (TransformerOptimus#1238) Bug Fix * Mixpanel integration (TransformerOptimus#1256) Mix Panel * Models Marketplace bug fix for dev (TransformerOptimus#1266) * Fix 1257 dev (TransformerOptimus#1269) Bug Fix * add cache layer (TransformerOptimus#1275) Added package caching for github actions workflow * Fix api dev (TransformerOptimus#1283) * save other config to agent_execution config * add config * mixpanel changes (TransformerOptimus#1285) * revert README.MD * rename error_handling.py to error_handler.py (TransformerOptimus#1287) --------- Co-authored-by: Rounak Bhatia <rounak@contlo.com> Co-authored-by: TransformerOptimus <muknrq@gmail.com> Co-authored-by: Abhijeet <129729795+luciferlinx101@users.noreply.github.com> Co-authored-by: Taran <97586318+Tarraann@users.noreply.github.com> Co-authored-by: Maverick-F35 <138012351+Maverick-F35@users.noreply.github.com> Co-authored-by: Rounak Bhatia <f20201807@goa.bits-pilani.ac.in> Co-authored-by: namansleeps <mandhani12@gmail.com> Co-authored-by: Aditya Sharma <138581531+AdityaSharma13064@users.noreply.github.com> Co-authored-by: Sayan Samanta <139119661+sayan1101@users.noreply.github.com> Co-authored-by: Kalki <97698934+jedan2506@users.noreply.github.com> Co-authored-by: Tarraann <jot.taran15522@gmail.com> Co-authored-by: Arkajit Datta <61142632+Arkajit-Datta@users.noreply.github.com> Co-authored-by: jagtarcontlo <123375045+jagtarcontlo@users.noreply.github.com> Co-authored-by: Captain Levi <123375045+CaptainLevi0007@users.noreply.github.com> Co-authored-by: namansleeps <122260931+namansleeps@users.noreply.github.com> Co-authored-by: I’m <133493246+TransformerOptimus@users.noreply.github.com> Co-authored-by: Jagtar Saggu <jagtarsaggu@Jagtars-MacBook-Pro.local>

* fix * fixing the toolkit config in iteration workflow * List file s3 fix (#1076) * List File S3 * Unit Test Added --------- Co-authored-by: Taran <97586318+Tarraann@users.noreply.github.com> * workflow changes * minor seed file fix * frontend fixes (#1079) * fixed api_bug (#1080) * add one condition (#1082) * api fix (#1087) * Tools error fix (#1093) * webhooks frontend + api calls complete almost * Tool-LTM(Updated) (#1039) * Toolkit configuration fix (#1102) * webhooks compplete frontend * schedule agent fix (#1104) Co-authored-by: Rounak Bhatia <rounak@contlo.com> * Models superagi (#936) Models Superagi * Models superagi (#1108) Bug Fixes * Changes for no receiver address * made changes to github helper * Models superagi (#1112) * Models superagi (#1117) * Models fixes (#1118) * Models Frontend Changes * Models Frontend Changes * Models Frontend Changes * \n bug resolved (#1122) * PDF and DOCX support in Write File - Feature Improvement, close #548 (#928) * Added functions to write various file types and a file handler too * FileManager updated to handle and save HTMLs. * adding PDF + DOCX support to save images * Added Wkhtmltopdf package installation run commands in docker * Added get_all_responses feature for extractng the response for particular tools * Added Image embedding feature, this will extract and embed the images generated by the agent during the run * renaming functions and refactoring * renaming functions and refactoring * Update Dockerfile * removing unsused classmethods * Finding generated images and attached files in the write tool. Images are fetched inorder to be embedded in the respective file type. * Adding the filename and paths to the Resource Manager S3 storage * Code Cleanup * added logger: Fix for the failing TEST --------- Co-authored-by: Fluder-Paradyne <121793617+Fluder-Paradyne@users.noreply.github.com> * Revert "PDF and DOCX support in Write File - Feature Improvement, close #548 (#928)" (#1124) This reverts commit 8b01357. * expose port * latest safetensors breaking in macs (#1134) * Changes in save template (#1120) * fix * changes in save template * changes in save template * bug fixes * changes in save template * changes in save template --------- Co-authored-by: Rounak Bhatia <rounak@contlo.com> Co-authored-by: namansleeps <mandhani12@gmail.com> * Main to dev sync (#1139) * Models fixes (#1126) * Models Frontend Changes * Models Frontend Changes * Models Frontend Changes * Backend Compatibility for New/Existing users on local * DEV api key requirements * removing print statements * removing print statements * removing print statements * removing print statements * backend compatibility * backend compatibility * backend compatibility * added filters in the webhooks * fix * added filters in the webhooks * Models fixes (#1145) Fixes related to Models Feature * Jira Bug Fix * Jira Bug Fix 2.0 * Jira Bug Fix 3.0 * added filters in the webhooks * Models fixes (#1147) Model Feature Fixes * Bug fix model redirection (#1148) Bug Fix - Model URL Redirection * added tool config for dalle * removed model dependency on dalle tool * Remove hardcoded creds * fixed env error * removed refactoring from main * removed refactoring * removed refactoring * handled error * stop agent from executing if model is not found (#1156) * entity details (#1158) * Metric frontend (#1152) * added filters in webhooks * added filters in webhooks * minor changes * webhooks complete * minor changes for PR * minor changes for PR * Publish agent template to marketplace (#1106) * publish agent to marketplace --------- Co-authored-by: Rounak Bhatia <rounak@contlo.com> Co-authored-by: namansleeps <mandhani12@gmail.com> * added filters in webhooks * added filters in webhooks * resolving conflicts * added filters in the webhooks * lint issue fixed * bug fix of prev PR * fix for new run and edit agent * error handling * added filters in webhooks * fix for knowledge search tool * Docker digitalocean deployment * changed branch name * added filters in the webhooks * changes * removed region * added button * change in branch * added filters in the webhooks * Update conftest.py * Added filters in the webhooks (#1140) * webhooks frontend + web hooks with filters --------- Co-authored-by: namansleeps <mandhani12@gmail.com> Co-authored-by: Fluder-Paradyne <121793617+Fluder-Paradyne@users.noreply.github.com> * Models calls logs dev (#1174) Call logs organisation level bug fix * models scroll fix, format of log timestamp fix, adding of loader to models, toolkit metrics dropdown bug fixed, publish agent dropdown bug (#1171) * Update app.yaml (#1179) * fixes related to webhooks * fixes for webhooks * Fixes for webhooks (#1181) * fixes for webhooks --------- Co-authored-by: namansleeps <mandhani12@gmail.com> Co-authored-by: Fluder-Paradyne <121793617+Fluder-Paradyne@users.noreply.github.com> * bugs by qa (#1178) * Fix for schedule agent (#1184) Co-authored-by: Tarraann <jot.taran15522@gmail.com> * Entity fix (#1185) * fixes for webhooks * fixes for webhooks * fix added for index state (#1188) * fix added for index state * Update KnowledgeTemplate.js --------- Co-authored-by: Tarraann <jot.taran15522@gmail.com> * API bug fixes for SDK (#1189) * fix api's for sdk * removed unused imports --------- Co-authored-by: jagtarcontlo <123375045+jagtarcontlo@users.noreply.github.com> * Main to dev sync v12 (#1193) sync back to dev ------ Co-authored-by: Taran <97586318+Tarraann@users.noreply.github.com> Co-authored-by: TransformerOptimus <muknrq@gmail.com> Co-authored-by: Fluder-Paradyne <121793617+Fluder-Paradyne@users.noreply.github.com> Co-authored-by: Maverick-F35 <138012351+Maverick-F35@users.noreply.github.com> Co-authored-by: BoundlessAsura <122777244+boundless-asura@users.noreply.github.com> Co-authored-by: Akshat Jain <92881074+Akki-jain@users.noreply.github.com> Co-authored-by: sayan1101 <139119661+sayan1101@users.noreply.github.com> Co-authored-by: Rounak Bhatia <f20201807@goa.bits-pilani.ac.in> Co-authored-by: Rounak Bhatia <rounak@contlo.com> Co-authored-by: Kalki <97698934+jedan2506@users.noreply.github.com> Co-authored-by: Tarraann <jot.taran15522@gmail.com> Co-authored-by: rakesh-krishna-a-s <akrishna@contlo.com> Co-authored-by: Captain Levi <123375045+CaptainLevi0007@users.noreply.github.com> Co-authored-by: andrew-kelly-neutralaiz <128111428+andrew-kelly-neutralaiz@users.noreply.github.com> Co-authored-by: James Wakelim <james.wakelim@neutralaiz.com> * added button * GitHub pull request tools (#1190) * adding github review tools * cleanup and adding code review prompt * fixing comments * PDF and DOCX support in Write File - Feature Improvement, close #548 (#1125) Co-authored-by: Fluder-Paradyne <121793617+Fluder-Paradyne@users.noreply.github.com> Co-authored-by: Abhijeet <129729795+luciferlinx101@users.noreply.github.com> * minor documentation fix * Design bugs (#1199) * fetching token limit from db * Revert "PDF and DOCX support in Write File - Feature Improvement, close #548 (#1125)" (#1202) This reverts commit 26f6a1d. * Unit Test Fix (#1203) * adding of docs and and discord link correction (#1205) * openai error handling * error_handling * api call only when agent is running * Feature : Wait block for agent workflow (#1186) Agent Wait Block Step * minor changes (#1213) Co-authored-by: Jagtar Saggu <jagtarsaggu@Jagtars-MacBook-Pro.local> * error handling * error handling * error handling * error handling * error handling * fix * fix * fix * error handling * models changes (#1207) Model related frontend changes. * error handling * models marketplace changes (#1219) Co-authored-by: Abhijeet <129729795+luciferlinx101@users.noreply.github.com> * minor changes * error handling * error handling * removing single qoutes (#1224) Co-authored-by: namansleeps <mandhani12@gmail.com> * apm changes (#1222) APM Bug Fixes * list tool fix * list tool fix * PR CHANGES * entity fix for dev (#1230) Entity Fix for Dev * frontend changes (#1231) * read_tool_fix * fix * waiting block frontend (#1233) Waiting Block Changes and Frontend Addition * Dev Fixes (#1242) * read tool fix * Maintaining dev (#1244) Dev Fix * added logs (#1246) * error_handling fix (#1247) Error handling fix. * Feature first login src (#1241) Adding source in user database for analytics. * apollo NoneType bug fix (#1238) Bug Fix * Mixpanel integration (#1256) Mix Panel * Models Marketplace bug fix for dev (#1266) * Fix 1257 dev (#1269) Bug Fix * add cache layer (#1275) Added package caching for github actions workflow * Fix api dev (#1283) * save other config to agent_execution config * add config * mixpanel changes (#1285) * rename error_handling.py to error_handler.py (#1287) * Analytics login (#1258) * calendar issues fixed * append fle tool bug fixed (#1294) Co-authored-by: Tarraann <jot.taran15522@gmail.com> * adding cookie in access token (#1301) * local_llms * local_llms * local_llms * local_llms * local_llms * fixes * models error fixed (#1308) * local_llms * local_llms * local_llms * local_llms * local_llms * frontend_changes * local_llms * local_llms * local_llms * local_llms * local_llms_frontend * fixes * fixes * fixes * fixes * merged main into local_llm_final * merged main into local_llm_final * local llms --------- Co-authored-by: TransformerOptimus <muknrq@gmail.com> Co-authored-by: Abhijeet <129729795+luciferlinx101@users.noreply.github.com> Co-authored-by: Taran <97586318+Tarraann@users.noreply.github.com> Co-authored-by: Fluder-Paradyne <121793617+Fluder-Paradyne@users.noreply.github.com> Co-authored-by: Maverick-F35 <138012351+Maverick-F35@users.noreply.github.com> Co-authored-by: namansleeps <mandhani12@gmail.com> Co-authored-by: Aditya Sharma <138581531+AdityaSharma13064@users.noreply.github.com> Co-authored-by: Sayan Samanta <139119661+sayan1101@users.noreply.github.com> Co-authored-by: Kalki <97698934+jedan2506@users.noreply.github.com> Co-authored-by: Tarraann <jot.taran15522@gmail.com> Co-authored-by: Arkajit Datta <61142632+Arkajit-Datta@users.noreply.github.com> Co-authored-by: rakesh-krishna-a-s <akrishna@contlo.com> Co-authored-by: jagtarcontlo <123375045+jagtarcontlo@users.noreply.github.com> Co-authored-by: Captain Levi <123375045+CaptainLevi0007@users.noreply.github.com> Co-authored-by: namansleeps <122260931+namansleeps@users.noreply.github.com> Co-authored-by: I’m <133493246+TransformerOptimus@users.noreply.github.com> Co-authored-by: Jagtar Saggu <jagtarsaggu@Jagtars-MacBook-Pro.local> Co-authored-by: Ubuntu <ubuntu@ip-10-14-2-4.ec2.internal>

Arkajit-Datta and others added 18 commits July 21, 2023 18:52

Added functions to write various file types and a file handler too

5e80166

FileManager updated to handle and save HTMLs.

2738a9a

adding PDF + DOCX support to save images

0fdeff9

Merge branch 'dev' into feature/write-file-pdf-docx-support

608f738

Added Wkhtmltopdf package installation run commands in docker

a0ca125

Added get_all_responses feature for extractng the response for partic…

cccd599

…ular tools

Added Image embedding feature, this will extract and embed the images…

285a132

… generated by the agent during the run

Merge branch 'feature/write-file-pdf-docx-support' into dev

610bd20

renaming functions and refactoring

74c938c

renaming functions and refactoring

1c4bbba

Merge branch 'feature/write-file-pdf-docx-support' into dev

0fa209f

Update Dockerfile

5803171

removing unsused classmethods

31b20a6

Finding generated images and attached files in the write tool. Images…

9e2b4d6

… are fetched inorder to be embedded in the respective file type.

Merge branch 'dev' into dev

eb430e6

Adding the filename and paths to the Resource Manager S3 storage

a24df27

Code Cleanup

0dc6fa6

added logger: Fix for the failing TEST

3c0a578

luciferlinx101 reviewed Aug 25, 2023

View reviewed changes

Arkajit-Datta added 2 commits August 26, 2023 16:03

Exceptions dir created to store custom exceptions

4993251

review changes

677c28e

luciferlinx101 reviewed Aug 29, 2023

View reviewed changes

superagi/exceptions/file_errors.py Outdated Show resolved Hide resolved

Dockerfile Show resolved Hide resolved

Arkajit-Datta added 4 commits September 4, 2023 13:33

updated the name of exception file

6606919

update exception import

91adecf

Added unit test cases for file manager

224b227

Added CSV test file UT case

82ce44b

code cleanup

7cf317d

luciferlinx101 requested changes Sep 6, 2023

View reviewed changes

superagi/models/agent_execution_feed.py Outdated Show resolved Hide resolved

superagi/tools/tool_response_query_manager.py Outdated Show resolved Hide resolved

superagi/tools/image_generation/stable_diffusion_image_gen.py Outdated Show resolved Hide resolved

Arkajit-Datta added 3 commits September 6, 2023 13:47

reverting the few unecessary changes

6f4fdf5

Merge branch 'develop' into dev

694477d

removing changes

7c834c0

Arkajit-Datta requested a review from luciferlinx101 September 6, 2023 08:41

nihiragarwal24 approved these changes Sep 6, 2023

View reviewed changes

Merge branch 'dev' into dev

6275fba

luciferlinx101 approved these changes Sep 6, 2023

View reviewed changes

luciferlinx101 merged commit 26f6a1d into TransformerOptimus:dev Sep 6, 2023
2 of 5 checks passed

luciferlinx101 added a commit that referenced this pull request Sep 7, 2023

Revert "PDF and DOCX support in Write File - Feature Improvement, close

d13c57a

#548 (#1125)" This reverts commit 26f6a1d.

luciferlinx101 mentioned this pull request Sep 7, 2023

Revert "PDF and DOCX support in Write File - Feature Improvement, close #548" #1202

Merged

luciferlinx101 added a commit that referenced this pull request Sep 7, 2023

Revert "PDF and DOCX support in Write File - Feature Improvement, close

3153def

#548 (#1125)" (#1202) This reverts commit 26f6a1d.

luciferlinx101 added a commit that referenced this pull request Sep 12, 2023

Revert "Revert "PDF and DOCX support in Write File - Feature Improvem…

a8ba439

…ent, close #548 (#1125)" (#1202)" This reverts commit 3153def.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PDF and DOCX support in Write File - Feature Improvement, close #548 #1125

PDF and DOCX support in Write File - Feature Improvement, close #548 #1125

Arkajit-Datta commented Aug 25, 2023 •

edited

Loading

luciferlinx101 Aug 25, 2023

Arkajit-Datta Aug 26, 2023

luciferlinx101 Aug 25, 2023

Arkajit-Datta Aug 26, 2023

luciferlinx101 Aug 25, 2023

Arkajit-Datta Aug 26, 2023

luciferlinx101 Aug 25, 2023

Arkajit-Datta Aug 26, 2023

luciferlinx101 Aug 25, 2023

Arkajit-Datta Aug 26, 2023

luciferlinx101 Aug 25, 2023

Arkajit-Datta Aug 26, 2023

codecov bot commented Aug 26, 2023 •

edited

Loading

Arkajit-Datta commented Aug 29, 2023

Arkajit-Datta commented Aug 30, 2023

luciferlinx101 commented Aug 31, 2023

Arkajit-Datta commented Sep 4, 2023

		@@ -47,12 +47,14 @@ def get_last_tool_response(cls, session: Session, agent_execution_id: int, tool_
		AgentExecutionFeed.role == "system").order_by(AgentExecutionFeed.created_at.desc()).all()

		for agent_execution_feed in agent_execution_feeds:

		@@ -11,7 +11,7 @@ def __init__(self, session: Session, agent_execution_id: int,memory:VectorStore)

		final_path = ResourceHelper.get_resource_path(file_name)

		logger.info(f"{file_name} - File written successfully")

PDF and DOCX support in Write File - Feature Improvement, close #548 #1125

PDF and DOCX support in Write File - Feature Improvement, close #548 #1125

Conversation

Arkajit-Datta commented Aug 25, 2023 • edited Loading

Description

Related Issues

Solution and Design

Test Plan

Type of change

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Aug 26, 2023 • edited Loading

Codecov Report

Arkajit-Datta commented Aug 29, 2023

Arkajit-Datta commented Aug 30, 2023

luciferlinx101 commented Aug 31, 2023

Arkajit-Datta commented Sep 4, 2023

Arkajit-Datta commented Aug 25, 2023 •

edited

Loading

codecov bot commented Aug 26, 2023 •

edited

Loading