Expand databricks agent error handling to fallback to token limit errors

[zanesq]

I noticed the UI shows a generic 401 unauthorized (occasionally) or 400 bad request from Databricks when a token limit is reached instead of kicking off the auto summarization. It doesn't recognize that its a token limit error because the message doesn't have the phrases we look for like "token limit" in the error message. It appears to be a token limit issue because manually using the summarize feature allows chat to continue while just sending a chat message without summarization does not.

This adds fallback logic to work around that and allows the UI to handle it gracefully by allowing chat to continue with auto summarize.

[DOsinga]

thanks for looking into this. I don't think we should be adding more ad hoc checks like this to providers though. my approach is this: #3558 -- add universal error parsing and retry logic to all providers.

[zanesq]

Makes sense, I'll close for now. Lets see if this fixes the issue and we might not need this.