Generative AI Language Modeling with Transformers
My twelfth module in my IBM course!
💡
What I learned!
- Learning how transformers process sequential data using positional encoding and attention mechanisms;
- Exploring how to implement positional encoding in PyTorch and understand how attention helps models focus on relevant...;
- Learning how decoder-based models like GPT are trained using causal language modeling and implemented in PyTorch for both training and inference;
- Exploring encoder-based models, such as Bidirectional Encoder Representations from Transformers (BERT), and understand their pretraining strategies using masked language modeling (MLM) and next sentence prediction (NSP), along with data preparation techniques in PyTorch.
Subscribe to my monthly newsletter
No spam, no sharing to third party. Only you and me.
Member discussion