Add support to different models to num_tokens_from_text

[thinkall]

Currently only gpt-x models are supported, would be nice to add support to more models.

[vidhula17]

I would like to work on this. Can you please assign this to me?

[thinkall]

I would like to work on this. Can you please assign this to me?

Hi @vidhula17 , thanks for your interest, I’ve assigned the task to you. Let me know if you need any assistance.

[vidhula17]

I have created a PR for this issue. Kindly look into it.