feat: add support for IBM watsonx.ai models

[rahulkalluri]

Adding using the ibm-watsonx-ai package as another LLM method. At this time, the watsonx.ai package does not support multi-turn conversations (I've included that in the updated docs).

• Tests added and passed if fixing a bug or adding a new feature.
• All code checks passed.
• closes Support for watsonx.ai natively or through Langchain #942

[gventuri]

Thanks a lot @rahulkalluri. Tests are failing, can you check it out?

[rahulkalluri]

Thanks a lot @rahulkalluri. Tests are failing, can you check it out?

After some quick investigation it looks like the ibm-watsonx-ai package requires python 3.10 and up. Would it be okay with updating the pandasai requirements to minimum of python 3.10?

[gventuri]

@rahulkalluri I think this is something we need to discuss with the community. But since it's an optional dependency, you can make it only available for some python versions:

For example:

sqlalchemy-bigquery = {version = "^1.8.0", optional = true, markers = "python_version >= '3.8' and python_version < '3.13'"}

Hope it helps

[mspronesti]

Isn't the langchain integration enough?

[rahulkalluri]

Isn't the langchain integration enough?

I saw other users were having issues using the langchain integration, plus I thought having a more native integration would be easier to use.

[mspronesti]

When I tried using it for IBM Watson (a couple of months ago) I didn't run into any issues. Did you face any?

[rahulkalluri]

When I tried using it for IBM Watson (a couple of months ago) I didn't run into any issues. Did you face any?

To be honest I tried and couldn't figure out how to leverage langchain with pandas and watsonx.

On top of that, the package the documentation refers to is now deprecated and is superseded by ibm-watsonx-ai which I've included here.

Since I wasn't able to figure it out and I saw the open issue, I thought it would be best to have a more native integration with watsonx, similar to other vendors like Amazon/Google.

[gventuri]

@rahulkalluri almost there, could you fix the linting errors? Also, what issue where you facing with LangChain?

[rahulkalluri]

@rahulkalluri almost there, could you fix the linting errors? Also, what issue where you facing with LangChain?

Sorry about that! Ran through the tests, we should be good to go now.

On the LangChain stuff, I think it's more user error than anything. Nothing was broken, and I'll post in the Discord for some help if need be.

[rahulkalluri]

Hey @gventuri I see the test failed on Python 3.9 which makes sense since the ibm-watsonx-ai package only works on 3.10 and above. Are you able to trigger the CI builds for 3.10+ only?

[gventuri]

@rahulkalluri can we temporarily use a skipif for the old version?

i.e.

@pytest.mark.skipif(sys.version_info[:2] != (3, 8), reason="requires Python 3.8")

What do you think? We don't want to drop the support for python 3.8 yet!

[rahulkalluri]

@rahulkalluri can we temporarily use a skipif for the old version?

i.e.

@pytest.mark.skipif(sys.version_info[:2] != (3, 8), reason="requires Python 3.8")

What do you think? We don't want to drop the support for python 3.8 yet!

Absolutely, PandasAI should continue to support Python 3.9. I was not aware of the pytest skip but I've implemented that alongside some other minor changes to ensure users won't have issues if they're using Python 3.9.
I tested the code on my machine in both Python 3.9 and 3.10 environments as well.

Thanks for the suggestion!

[gventuri]

@rahulkalluri thanks a lot for the effort, merging!

[raviisrani]

@rahulkalluri thanks a lot for the effort, merging!

Does not work still..
I tried testing a demo script with watsonx... I have updated my python all the way from 3.10 to 3.12.. still does not work on any
version.

One observation, it shows working sometimes when you try to first run the script with other platform like gemini.. the results are updated into the cache and then if you run with watsonx, it does not run. I tried to remove all cache and log files.. then u get the error. Suggest testing it in a clean folder.

Sending the code, data file, log file for your analysis... renamed the python script as testpa-wx.py.txt
pandasai.log
supreme.csv
testpa-wx.py.txt

[SebG-js]

This tool https://pypi.org/project/ibm-watsonx-ai doesn't support py3.12, and maybe 3.11
See this ticket :
#1386

Could we remove the ibm-watsonx-ai support and got the py3.12 support ? @rahulkalluri @gventuri