How to make an assistant call an API POST query including a file attachment ?

[djuillard]

I am planning to build an assistant that will use an API service. This API service expects that we send it, through a POST query, a file attachment. I am wondering how we can make it work, given that every file loaded into the chatbot is immediately embedded and stored into the vector database.

Concretely, I would need to de-activate temporarily the RAG and make sure my assistant can read the file to POST it to the API.

Is that feasible today, or would it require an adaptation and extra-features ?

[djuillard]

Hi,

Let me raise this up again and be more accurate.

When we upload a file within the prompt field, the file is chunked and vectorized into the vector database. But does that file remain somewhere in the application stored as a real file in the filesystem ? If so, is there a "clean" way to access it and make some processing on it through an assistant ? I would like to send that real file into a POST API query.

Thanks

[danny-avila]

the OpenAI API, including Assistants API does not accept files as direct input. Assistants accepts files that go into their "tool_resources" buckets, which can either be file search or code interpreter. I'm developing something similar for Agents. But also I think what you're getting at is being able to upload files as context input, which is discussed here: #2755

And here for Claude, which does accept certain files (PDF) as input: #4616

[djuillard]

Thanks a lot Danny for your answer. I am aware of what you are working on about Agents, and am lokking forward to using it in prod environment.
What I am looking for here is getting able to use third party API service like https://github.com/DS4SD/docling to get a proper reading of a file, prior to embedd it into RAG. And I was thinking that an assistant or maybe your forthcoming Agent implementation could do this preliminary step.

[danny-avila]

Interesting, cool to see a project by IBM/DeepSearch. I may integrate Docling to the RAG API to get better readings and facilitate the text conversion as well