Talk: Bespoke LLMs - Building and Scaling customized large language models

Slides and code for the talk at Analytics Vidhya's DHS 2023 event and Saama's Tech meetup on 29th June 2023.

I spoke about the benefits of building custom LLMs by adapting open source models for enterprises, including the process ( data collection, preprocessing, and training ) and tools & techniques for a successful adaptation.

Code:

Prompt-tuning:

Demo of prompt tuning with wikisql dataset. Santacoder is prompt-tuned on NL2SQL task.

Evol Instruction tuning:

Instruction tuning Santacoder model on nickrosh/Evol-Instruct-Code-80k-v1 using PEFT library.

Slides:

• Google DOC : link
• PDF : /slides/Bespoke LLMs _ Building and Scaling customized large language models.pdf