Coding Transformers, the framework that ChatGPT is built on, from Scratch
Thursday, 16. May 2024 | 08:30 - 17:00
Description
We'll learn how to code, train and use transformers from scratch in Python with PyTorch and Lightning. We'll then leverage premade transformer classes to build, train and use a Large Language Model (LLM) Transformer that can answer basic questions.
Topics
Step-by-step overviews of how transformers work
An introduction to Lightning Studios for easy access cloud computing
Coding a transformer from scratch, including word embedding, positional encoding, self-attention, masked self-attention, and encoder-decoder attention.
Training our transformer with a simple dataset to translate English to Spanish.
Using our transformer to translate user input.
Downloading and fine-tuning an open source large language model.
Using the large language model to answer basic questions.
Requirements
Basic coding skills in Python.
Target audience
Anyone who wants to learn how artificial intelligence (AI) like ChatGPT works. No previous experience with AI is necessary.
