[Bug] Can't decode DOC files

[amomra]

Context / Scenario

The user upload a document with the DOC format

What happened?

When the user upload a document with the legacy DOC format, the application throws an exception saying that the file is corrupt. Analyzing the logs is shown that the application is trying to parse it as a DOCX file.

Importance

I cannot use Kernel Memory

Platform, Language, Versions

It happens in Windows and Linux in dotnet 8

Relevant log output

System.IO.FileFormatException: File contains corrupted data.
   at System.IO.Packaging.ZipPackage..ctor(Stream s, FileMode packageFileMode, FileAccess packageFileAccess)
   at System.IO.Packaging.Package.Open(Stream stream, FileMode packageMode, FileAccess packageAccess)
   at DocumentFormat.OpenXml.Packaging.PackageLoader.OpenCore(Stream stream, Boolean readWriteMode)
   at DocumentFormat.OpenXml.Packaging.WordprocessingDocument.Open(Stream stream, Boolean isEditable, OpenSettings openSettings)
   at DocumentFormat.OpenXml.Packaging.WordprocessingDocument.Open(Stream stream, Boolean isEditable)
   at Microsoft.KernelMemory.DataFormats.Office.MsWordDecoder.DocToText(Stream data) in /sistemas/projetos/teste-llama/kernel-memory/kernel-memory/service/Core/DataFormats/Office/MsWordDecoder.cs:line 28
   at Microsoft.KernelMemory.DataFormats.Office.MsWordDecoder.DocToText(BinaryData data) in /sistemas/projetos/teste-llama/kernel-memory/kernel-memory/service/Core/DataFormats/Office/MsWordDecoder.cs:line 23
   at Microsoft.KernelMemory.Handlers.TextExtractionHandler.ExtractTextAsync(FileDetails uploadedFile, BinaryData fileContent, CancellationToken cancellationToken) in /sistemas/projetos/teste-llama/kernel-memory/kernel-memory/service/Core/Handlers/TextExtractionHandler.cs:line 140
   at Microsoft.KernelMemory.Handlers.TextExtractionHandler.InvokeAsync(DataPipeline pipeline, CancellationToken cancellationToken) in /sistemas/projetos/teste-llama/kernel-memory/kernel-memory/service/Core/Handlers/TextExtractionHandler.cs:line 78
   at Microsoft.KernelMemory.Pipeline.DistributedPipelineOrchestrator.RunPipelineStepAsync(DataPipeline pipeline, IPipelineStepHandler handler, CancellationToken cancellationToken) in /sistemas/projetos/teste-llama/kernel-memory/kernel-memory/service/Core/Pipeline/DistributedPipelineOrchestrator.cs:line 214
   at Microsoft.KernelMemory.Pipeline.DistributedPipelineOrchestrator.<>c__DisplayClass3_0.<<AddHandlerAsync>b__0>d.MoveNext() in /sistemas/projetos/teste-llama/kernel-memory/kernel-memory/service/Core/Pipeline/DistributedPipelineOrchestrator.cs:line 148

[dluc]

@amomra how would you prefer handling the legacy DOC format? ignore and skip the files, logging an error maybe?

[amomra]

@dluc I observed that the app enters in an error loop when you uses RabbitMQ. This happens because the extractor throws an error with the DOC file, requeues the message and immediately retrieves it again causing the loop.
So, a least for now, I think DOC files should be ignored until there is a way to handle theses files since there isn't a portable way to do that (there is the Office Interop library but I think it works only in Windows).

[dluc]

The old format is now automatically ignored, unless a specific decoder is provided. The same approach is used for old formats of Word, Excel and PowerPoint