Notes

NLP

• Positional Encoding: one of the most clever engineering decisions in the Attention Is All You Need paperJune 3, 2026
• Building intuition for attention by understanding the limitations of RNNsMay 6, 2026
• Problems with pure RNNs and GRUs, and how LSTM tries to solve themMay 6, 2026
• The predecessors of Transformers: RNNs, how they work, and why attention was the missing blockMay 5, 2026
• Word representation using the common bag of words modelMay 5, 2026
• Stepping through the training of a neural network while building a sentiment classification modelMay 5, 2026
• Understanding the gradient flow in a computation graph by building a simple feedforward networkMay 5, 2026