Is there any possibility of optimizing the flair model (like INT8 quantization etc.,) ?

[abhipn]

I have been using Flair in our production environment for some time now, and I haven't faced any issues so far. But the issue here is not every organization uses GPU for inference, and having a CPU for inference will not be ideal when latency becomes important.

I was wondering is there a way I couldn't convert flair.pt to flair.onnx in the process to apply integer quantization, a small trade off of accuracy over performance is not actually a bad idea. I have gone through docs, but couldn't find any reference for optimizations or distillation etc.,

If someone managed to do it, really appreciate if you could share the details.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.